-- Logs begin at Thu 2022-09-01 13:05:33 UTC, end at Thu 2022-09-01 14:21:25 UTC. -- Sep 01 13:40:15.967833 dv-cinder-852210-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 13:40:15 2022] *** Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: nodename: dv-cinder-852210-7-iscsi Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: machine: x86_64 Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: clock source: unix Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: pcre jit disabled Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: detected number of CPU cores: 4 Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: current working directory: / Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: detected binary path: /usr/bin/uwsgi-core Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: your processes number limit is 47680 Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: your memory page size is 4096 bytes Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: detected max file descriptor number: 2048 Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: lock engine: pthread robust mutexes Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: thunder lock: enabled Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 13:40:15.979960 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 13:40:16.022637 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: Python main interpreter initialized at 0x556eaa4c79c0 Sep 01 13:40:16.022637 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: python threads support enabled Sep 01 13:40:16.022637 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: your server socket listen backlog is limited to 100 connections Sep 01 13:40:16.022637 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: your mercy for graceful operations on workers is 90 seconds Sep 01 13:40:16.023012 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 13:40:16.023080 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: *** Operational MODE: preforking *** Sep 01 13:40:16.023346 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: *** uWSGI is running in multiple interpreter mode *** Sep 01 13:40:16.023346 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: spawned uWSGI master process (pid: 88124) Sep 01 13:40:16.023596 dv-cinder-852210-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 13:40:16.024149 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: spawned uWSGI worker 1 (pid: 88133, cores: 1) Sep 01 13:40:16.024473 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: spawned uWSGI worker 2 (pid: 88134, cores: 1) Sep 01 13:40:16.025817 dv-cinder-852210-7-iscsi devstack@keystone.service[88124]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 13:40:20.384913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:40:20.384913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: from cryptography.utils import int_from_bytes Sep 01 13:40:20.385661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:40:20.385661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: from cryptography.utils import int_from_bytes Sep 01 13:40:20.385816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:40:20.385816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: from cryptography.utils import int_from_bytes Sep 01 13:40:20.386253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 13:40:20.386253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: from cryptography.utils import int_from_bytes Sep 01 13:40:20.437128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 13:40:20.437356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 13:40:20.437356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 13:40:20.437513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 13:40:20.437513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] command line args: [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 13:40:20.437633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] command line args: [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 13:40:20.437810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 13:40:20.437913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 13:40:20.437994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ================================================================================ {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 13:40:20.438070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ================================================================================ {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 13:40:20.438219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] admin_token = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.438305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] admin_token = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.438468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] config_dir = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.438552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] config_dir = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.438659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.438744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.438850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] config_source = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.438934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] config_source = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] debug = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] debug = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.439991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] list_limit = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] list_limit = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] log_config_append = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] log_config_append = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.440907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] log_dir = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] log_dir = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] log_file = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] log_file = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.441903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.442032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.442032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.442224 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.442305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.442378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.442443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.442618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443605 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.443993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.444967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.445091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.445243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.445365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.445734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.445937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.445937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] publish_errors = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] publish_errors = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.446955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] standard_threads = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] standard_threads = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.447908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] use_journal = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] use_journal = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] use_json = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] use_json = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.448831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] use_stderr = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.449021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] use_stderr = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.449143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] use_syslog = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.449257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] use_syslog = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.449369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.449486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 13:40:20.449605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.449730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.449872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.449987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.450932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451603 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.451947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.452990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.453921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.454132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.454244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.454361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.454551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.454666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.454785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.454903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455822 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.455943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.456901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.457901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.458952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459544 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.459876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.460920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.461965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.462884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.463919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.464990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.465111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.465225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.465338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.465450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.465562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.465675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.465881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.466940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.467993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.468926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.469985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.470097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.470327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.470456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.470571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.470685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.470799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.470912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.471970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.472903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.473969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] auth.external = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.474936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] auth.external = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] auth.password = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] auth.password = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.475975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] auth.token = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] auth.token = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.476909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.477927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] credential.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] credential.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.478979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.479104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.479220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.479334 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.479447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.479568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.479691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.479888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480351 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.480951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.481902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.482942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.483880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.484931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.485984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.486944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.487992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.488934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.489049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.489162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.489276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.489960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.490909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.491909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.492023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.492143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.492526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.492645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.492759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.492874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.492987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.493941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.494988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.495931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.496988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497102 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.497952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.498886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.499925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.500972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501544 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.501916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.502958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.503959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.504965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.505129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.505787 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.505907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.505907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506822 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.506937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.507986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.508913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] role.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] role.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] role.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] role.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.509932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510292 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.510985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.511924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.512959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.513987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514344 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.514923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.515955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516421 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.516884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.517987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.518923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.519968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.520922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.connection = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.521929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.connection = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.522806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.523941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.524852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.525904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 13:40:20.526020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.526020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 13:40:20.526020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 13:40:20.554980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.555211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.555439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.555524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.555741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.555972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.564215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.564420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.592828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.593005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.593090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.593244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.650251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.651085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.700070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.700305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.701014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.701253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.856909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.857146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.857382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.857503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.857613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.857694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.857865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.857954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.858088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.858174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.858333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.858550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.882318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.882493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.882600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.882729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.882812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.882955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.883810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.894810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.894907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.908264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.908393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.908501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.908631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.908709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.908774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.908932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.909022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.909147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.909229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.909361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.909442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.909586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.909668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 13:40:20.926438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.926651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:40:20.926769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.927004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:40:20.927137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.927249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:40:20.927364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.927479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.927591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 13:40:20.927744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.927857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.928010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.928273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.928385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.928562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.928852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.937996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.938261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.938346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.938541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.938636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.938813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.938894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.939100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.946405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.946677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.949295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.949565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.949647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.949880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.949964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.950173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.950254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.950461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.950549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.950727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.950816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.951955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.952111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.952238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.975643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.975742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.977116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.977200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.977409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.977490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.982055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.982149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.982345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.982440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.982633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.982715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.989114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.989276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.989448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.990148 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.994439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.994537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:20.994737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.994819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.995029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.995111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.995316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.995396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.995597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.995685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.995880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.996156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.996458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.997710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:20.998034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.007498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.013994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.014108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.014281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.014417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.018182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.018274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.021335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.021598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.021682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.021927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.026098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.026204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.029237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.029325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.032403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.032612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.032709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.032929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.033019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.033221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.033303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.033533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.033662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.033863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.033995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.034154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.050769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.051061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.051333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.051431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.051602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.051684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.051967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.052729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.059166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.059604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.060721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.061269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.062442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.062725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.062954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.063250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.065416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.065706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.065930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.066131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.066257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.066424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.066505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.066728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.066817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.067062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.078260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.078515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.078600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.078831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.078970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.079097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.079264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.079397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.079561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.079759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.079891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.080023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.080152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.080282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.096289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.096391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.099816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.099914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.102794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.102906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.104248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.104330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.105944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.106066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.106234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.106323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.106531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.106676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.113033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.113628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.114576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.115191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.117945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.118254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.122426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.122515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.122586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.122649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.122709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.122709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.126840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.127134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.127359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.127440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.127655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.127941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.134450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.134733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.135016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 13:40:21.135099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.135297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.135379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.135640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.135736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.135879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.135962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.136160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.136242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.136440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.136521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.136721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.136810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.137935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.138123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.138226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.138401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.138682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 13:40:21.173824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556eaa4c79c0 pid: 88133 (default app) Sep 01 13:40:21.173958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556eaa4c79c0 pid: 88134 (default app) Sep 01 13:40:21.177456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89472cbe-77b8-49c6-b78a-5dbd2783ab8e None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:21.177836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89472cbe-77b8-49c6-b78a-5dbd2783ab8e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:21.178119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89472cbe-77b8-49c6-b78a-5dbd2783ab8e None None] PATH_INFO: `/v3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:21.182578 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Thu Sep 1 13:40:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:28.932945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d128d-71da-4460-8e28-468e279d3769 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:28.933529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d128d-71da-4460-8e28-468e279d3769 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:28.933529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d128d-71da-4460-8e28-468e279d3769 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:28.937803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1/2] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:40:28.943267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:28.943766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:28.943859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:29.014347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: INFO dbcounter [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] Registered counter for database keystone Sep 01 13:40:29.037912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG oslo_db.sqlalchemy.engines [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88133) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 13:40:29.044513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writer thread running {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 13:40:29.231193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG passlib.handlers.bcrypt [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88133) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 13:40:29.231516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG passlib.handlers.bcrypt [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88133) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 13:40:29.290693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:29.292307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:29.423356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 2/3] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 13:40:29.898129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d128d-71da-4460-8e28-468e279d3769 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:29.966347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: INFO dbcounter [None req-4f9d128d-71da-4460-8e28-468e279d3769 None None] Registered counter for database keystone Sep 01 13:40:29.985036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG oslo_db.sqlalchemy.engines [None req-4f9d128d-71da-4460-8e28-468e279d3769 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88134) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 13:40:29.991272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writer thread running {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 13:40:30.109112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d128d-71da-4460-8e28-468e279d3769 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:30.113521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71822390-2d9b-40e5-a9c4-a48e8795625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:30.114713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71822390-2d9b-40e5-a9c4-a48e8795625c admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:30.114999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71822390-2d9b-40e5-a9c4-a48e8795625c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:30.115280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71822390-2d9b-40e5-a9c4-a48e8795625c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:30.124278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71822390-2d9b-40e5-a9c4-a48e8795625c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:30.177746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING keystone.common.rbac_enforcer.enforcer [None req-71822390-2d9b-40e5-a9c4-a48e8795625c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 13:40:30.178058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71822390-2d9b-40e5-a9c4-a48e8795625c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:30.187917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 2/4] 10.222.0.61 () {58 vars in 1324 bytes} [Thu Sep 1 13:40:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 13:40:30.194473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:30.208786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c6283-bc5e-4d14-9609-94b24f0e14b5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:30.213178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a8d7c-3449-421c-aba5-272fed27be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:30.214309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8d7c-3449-421c-aba5-272fed27be48 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:30.214591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8d7c-3449-421c-aba5-272fed27be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:30.214861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8d7c-3449-421c-aba5-272fed27be48 admin admin] PATH_INFO: `/v3/projects/17c0369ad3194a3b8c268accfe2208d0` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:30.216416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a8d7c-3449-421c-aba5-272fed27be48 admin admin] RBAC: Authorizing `identity:get_project(project_id=17c0369ad3194a3b8c268accfe2208d0)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:30.269617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING keystone.common.rbac_enforcer.enforcer [None req-243a8d7c-3449-421c-aba5-272fed27be48 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 13:40:30.269989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a8d7c-3449-421c-aba5-272fed27be48 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:30.274846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 3/5] 10.222.0.61 () {56 vars in 1207 bytes} [Thu Sep 1 13:40:30 2022] GET /identity/v3/projects/17c0369ad3194a3b8c268accfe2208d0 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:32.162522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471feaa7-ed33-4506-8ec8-d604e1204c7c None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:32.163056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471feaa7-ed33-4506-8ec8-d604e1204c7c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:32.163056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471feaa7-ed33-4506-8ec8-d604e1204c7c None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:32.167209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 3/6] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:32.172664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb97a19-fcb9-4f67-8838-5ebce048734c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:32.173079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb97a19-fcb9-4f67-8838-5ebce048734c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:32.173220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb97a19-fcb9-4f67-8838-5ebce048734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:32.194053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-9eb97a19-fcb9-4f67-8838-5ebce048734c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:32.195574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9eb97a19-fcb9-4f67-8838-5ebce048734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:32.203506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 4/7] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 13:40:32.474941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471feaa7-ed33-4506-8ec8-d604e1204c7c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:32.486944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471feaa7-ed33-4506-8ec8-d604e1204c7c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:32.491129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc786433-206f-496f-943e-1089cd77ea0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:32.491718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc786433-206f-496f-943e-1089cd77ea0d admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:32.491998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc786433-206f-496f-943e-1089cd77ea0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:32.492504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc786433-206f-496f-943e-1089cd77ea0d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:32.500050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc786433-206f-496f-943e-1089cd77ea0d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:32.501814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc786433-206f-496f-943e-1089cd77ea0d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:32.510366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 4/8] 10.222.0.61 () {58 vars in 1324 bytes} [Thu Sep 1 13:40:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 13:40:32.515822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb97a19-fcb9-4f67-8838-5ebce048734c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:32.525338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb97a19-fcb9-4f67-8838-5ebce048734c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:32.529689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657106a4-f239-4dbc-8825-6000b4cce546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:32.530351 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657106a4-f239-4dbc-8825-6000b4cce546 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:32.530630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657106a4-f239-4dbc-8825-6000b4cce546 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:32.530902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657106a4-f239-4dbc-8825-6000b4cce546 admin admin] PATH_INFO: `/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:32.532217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657106a4-f239-4dbc-8825-6000b4cce546 admin admin] RBAC: Authorizing `identity:get_user(user_id=2afeb1cf31eb47f9a0c823030df9f4cb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:32.534066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657106a4-f239-4dbc-8825-6000b4cce546 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:32.535587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 5/9] 10.222.0.61 () {56 vars in 1198 bytes} [Thu Sep 1 13:40:32 2022] GET /identity/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:34.512893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c1a91-75e0-4773-a5e4-293eaa7faab4 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:34.513432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c1a91-75e0-4773-a5e4-293eaa7faab4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:34.513432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c1a91-75e0-4773-a5e4-293eaa7faab4 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:34.514867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 5/10] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:34.520166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05163d6b-8471-464f-b467-7393ead50658 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:34.520598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05163d6b-8471-464f-b467-7393ead50658 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:34.520858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05163d6b-8471-464f-b467-7393ead50658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:34.539147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-05163d6b-8471-464f-b467-7393ead50658 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:34.540365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-05163d6b-8471-464f-b467-7393ead50658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:34.549153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7715fad-4d8d-4641-88d0-1a18d6830677 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:34.549438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7715fad-4d8d-4641-88d0-1a18d6830677 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:34.549755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7715fad-4d8d-4641-88d0-1a18d6830677 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:34.551138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 6/11] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:34.557023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:34.557312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:34.557594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:34.600170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG passlib.handlers.bcrypt [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88134) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 13:40:34.600661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG passlib.handlers.bcrypt [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88134) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 13:40:34.622141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 6/12] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:34.650009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:34.651626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:34.751152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 7/13] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:34.894100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05163d6b-8471-464f-b467-7393ead50658 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:34.918522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:34.997310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05163d6b-8471-464f-b467-7393ead50658 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.018938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:35.031598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.032238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.032523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.032864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.044402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:35.053285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:35.054427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:35.055483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 7/14] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:40:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:40:35.060420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:35.126073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3cf60-4654-47ad-b646-a8031aece258 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.128323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:35.152103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:35.162884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.163764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.164043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.164313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.166166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df3dc5-e1b9-4b38-94f9-92692bf14953 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.200172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.200791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.201451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.201724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.202416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:35.202717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:35.204171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:35.216082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 8/15] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:35.221812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:35.231275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:35.232781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:35.325926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e54cf4-8c0f-4034-8490-16a6f0553dbf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.338426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 8/16] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:40:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:40:35.344053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:35.346747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:35.362706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.363362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.363618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.363997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] PATH_INFO: `/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.365212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2afeb1cf31eb47f9a0c823030df9f4cb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:35.366697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:35.368494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 9/17] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:35 2022] GET /identity/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:35.374007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:35.388060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:35.442795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d4ed-3b57-457f-9091-5035075ec5be None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.474650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7530e-b52a-4012-af08-480b859a73c6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.477235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1344290-fc23-47fc-8416-319852915547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.478122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1344290-fc23-47fc-8416-319852915547 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.478407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1344290-fc23-47fc-8416-319852915547 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.478682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1344290-fc23-47fc-8416-319852915547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.497282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1344290-fc23-47fc-8416-319852915547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:35.498776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1344290-fc23-47fc-8416-319852915547 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:35.508150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.508786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.509068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.509342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.510490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:35.511931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:35.513465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 10/18] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-b1344290-fc23-47fc-8416-319852915547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:40:35.517110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:35.518482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:40:35.519409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:40:35.519409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:40:35.519409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:40:35.519409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 9/19] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:40:35.519739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:35.524284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1344290-fc23-47fc-8416-319852915547 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:35.579863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:35.605223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:35.624134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3654b-35c7-40dc-8b75-0eb4ee918039 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.625627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1344290-fc23-47fc-8416-319852915547 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:35.658992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc9f93-e95d-4824-9822-92a0347d82aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.659187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915847d6-082e-4b32-b136-770b438d3695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:35.659602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9f93-e95d-4824-9822-92a0347d82aa None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.659710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915847d6-082e-4b32-b136-770b438d3695 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:35.659885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9f93-e95d-4824-9822-92a0347d82aa None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.659995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915847d6-082e-4b32-b136-770b438d3695 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:35.660159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9f93-e95d-4824-9822-92a0347d82aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.660243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915847d6-082e-4b32-b136-770b438d3695 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:35.661439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9f93-e95d-4824-9822-92a0347d82aa None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.domain.id=default, user.id=2afeb1cf31eb47f9a0c823030df9f4cb) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:35.661541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915847d6-082e-4b32-b136-770b438d3695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:35.661743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9f93-e95d-4824-9822-92a0347d82aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:35.661856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915847d6-082e-4b32-b136-770b438d3695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:35.663176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915847d6-082e-4b32-b136-770b438d3695 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:35.663288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9f93-e95d-4824-9822-92a0347d82aa None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:35.670139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 11/20] 10.222.0.61 () {56 vars in 1322 bytes} [Thu Sep 1 13:40:35 2022] GET /identity/v3/role_assignments?user.id=2afeb1cf31eb47f9a0c823030df9f4cb&scope.domain.id=default&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:35.681929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 10/21] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:37.983072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc8d4-c8e1-45ae-939a-724b33defa0d None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:37.983610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc8d4-c8e1-45ae-939a-724b33defa0d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:37.983610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc8d4-c8e1-45ae-939a-724b33defa0d None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:37.985193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 12/22] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:37.997047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a11b76-0490-43a6-ba74-761d4b9e6ce4 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:37.997969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a11b76-0490-43a6-ba74-761d4b9e6ce4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:37.998468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a11b76-0490-43a6-ba74-761d4b9e6ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.014210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d26b4c-0a06-4313-9246-3d6583791c4b None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.014768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d26b4c-0a06-4313-9246-3d6583791c4b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.015256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d26b4c-0a06-4313-9246-3d6583791c4b None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.016767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 13/23] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:38.024113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d06039-10b7-4443-a14b-2915393bd83f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.024634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d06039-10b7-4443-a14b-2915393bd83f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.025081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d06039-10b7-4443-a14b-2915393bd83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.052425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-21a11b76-0490-43a6-ba74-761d4b9e6ce4 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:38.054065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-21a11b76-0490-43a6-ba74-761d4b9e6ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:38.071539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f8d06039-10b7-4443-a14b-2915393bd83f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:38.073168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f8d06039-10b7-4443-a14b-2915393bd83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:38.210057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 11/24] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:38.211690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9acfa-961a-4a45-83c7-62fe9d2740a5 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.212163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9acfa-961a-4a45-83c7-62fe9d2740a5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.212798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9acfa-961a-4a45-83c7-62fe9d2740a5 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.214261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 12/25] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:38.216136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 14/26] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:38.238173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abc253-c674-4f02-8238-feeeb2fb3746 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.238714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abc253-c674-4f02-8238-feeeb2fb3746 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.239168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abc253-c674-4f02-8238-feeeb2fb3746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.267282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c8abc253-c674-4f02-8238-feeeb2fb3746 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:38.268757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c8abc253-c674-4f02-8238-feeeb2fb3746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:38.338197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:38.352558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 13/27] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:38.634934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d06039-10b7-4443-a14b-2915393bd83f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:38.728416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:38.763796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aec94-3fdd-4eb5-a8ae-9d7ae1813d7b None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.764484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aec94-3fdd-4eb5-a8ae-9d7ae1813d7b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.765046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aec94-3fdd-4eb5-a8ae-9d7ae1813d7b None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.766819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 14/28] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:38.767810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aec94-3fdd-4eb5-a8ae-9d7ae1813d7b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:38.780521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d06039-10b7-4443-a14b-2915393bd83f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:38.821754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:38.822399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.822674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.822943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.823631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:38.825025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:38.896466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2aec94-3fdd-4eb5-a8ae-9d7ae1813d7b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:38.935143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3936e-c76a-472e-b265-9da4fab22a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:38.936333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3936e-c76a-472e-b265-9da4fab22a99 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.936878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3936e-c76a-472e-b265-9da4fab22a99 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.937394 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3936e-c76a-472e-b265-9da4fab22a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.949526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3936e-c76a-472e-b265-9da4fab22a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:38.951610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3936e-c76a-472e-b265-9da4fab22a99 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-fba3936e-c76a-472e-b265-9da4fab22a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:40:38.968895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:38.969843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:40:38.971284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:40:38.971284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:40:38.971284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:40:38.973035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 15/29] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:38.974746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437b60e-272d-48ce-a953-727170c45c6e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:38.975247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437b60e-272d-48ce-a953-727170c45c6e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:38.975728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437b60e-272d-48ce-a953-727170c45c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:38.995813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a437b60e-272d-48ce-a953-727170c45c6e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:38.997398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a437b60e-272d-48ce-a953-727170c45c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:39.026042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:39.059135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 15/30] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Sep 1 13:40:38 2022] POST /identity/v3/roles => generated 219 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:39.060378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:39.086488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 16/31] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:39.088178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437b60e-272d-48ce-a953-727170c45c6e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:39.261927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca467e-ce6f-41d5-89e1-c345c3b96760 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:39.298708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f2e8e-9f71-4292-b7a8-d93f6d26b1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:39.299309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437b60e-272d-48ce-a953-727170c45c6e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:39.299942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f2e8e-9f71-4292-b7a8-d93f6d26b1e1 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.300412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f2e8e-9f71-4292-b7a8-d93f6d26b1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.300974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f2e8e-9f71-4292-b7a8-d93f6d26b1e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.305266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:39.317566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f2e8e-9f71-4292-b7a8-d93f6d26b1e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-229f2e8e-9f71-4292-b7a8-d93f6d26b1e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:39.323379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:39.328202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 16/32] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:40:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:39.328978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f8b0b-cd00-402d-85c5-d4370556388a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.330232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f8b0b-cd00-402d-85c5-d4370556388a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.330232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f8b0b-cd00-402d-85c5-d4370556388a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.334061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 17/33] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:39.334960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f8b0b-cd00-402d-85c5-d4370556388a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:39.343705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce7fd3-818d-48dc-9704-eb45112187d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:39.344330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7fd3-818d-48dc-9704-eb45112187d5 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.344612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7fd3-818d-48dc-9704-eb45112187d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.344894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7fd3-818d-48dc-9704-eb45112187d5 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.345553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce7fd3-818d-48dc-9704-eb45112187d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:39.345885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce7fd3-818d-48dc-9704-eb45112187d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:39.347289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce7fd3-818d-48dc-9704-eb45112187d5 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:39.359389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 17/34] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:39.360562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395a5b-70e2-4bc4-b738-8d602576b270 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.360849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395a5b-70e2-4bc4-b738-8d602576b270 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.361127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395a5b-70e2-4bc4-b738-8d602576b270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.385815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-6f395a5b-70e2-4bc4-b738-8d602576b270 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:39.387387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6f395a5b-70e2-4bc4-b738-8d602576b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:39.401643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:39.459044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f8b0b-cd00-402d-85c5-d4370556388a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:39.463912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 18/35] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:39.464747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395a5b-70e2-4bc4-b738-8d602576b270 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:39.495635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:39.496252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.496533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.496803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.497482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:39.498286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:39.500109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:39.513419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 18/36] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:39.518846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:39.585016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395a5b-70e2-4bc4-b738-8d602576b270 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:39.591429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:39.623972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269367-0afa-4b4a-9bb0-a07a2c283506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:39.624531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269367-0afa-4b4a-9bb0-a07a2c283506 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.624867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269367-0afa-4b4a-9bb0-a07a2c283506 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.625141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269367-0afa-4b4a-9bb0-a07a2c283506 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.626110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269367-0afa-4b4a-9bb0-a07a2c283506 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:39.627848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269367-0afa-4b4a-9bb0-a07a2c283506 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:39.640686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929feb87-8c53-4a35-a0c5-05ff02746773 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:39.676833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:39.677409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.677687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.678003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.678651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:39.680040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:39.774388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:39.811640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 19/37] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Sep 1 13:40:39 2022] POST /identity/v3/roles => generated 225 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:39.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:39.889653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 19/38] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:40:39 2022] POST /identity/v3/projects => generated 301 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:39.891282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a19e1d-cdfc-45b7-8808-50dce85f7e48 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:39.891571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a19e1d-cdfc-45b7-8808-50dce85f7e48 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:39.891852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a19e1d-cdfc-45b7-8808-50dce85f7e48 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:39.893279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 20/39] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:39.894290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a19e1d-cdfc-45b7-8808-50dce85f7e48 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:40.045428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a19e1d-cdfc-45b7-8808-50dce85f7e48 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:40.084340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639830e-2e4a-4012-a4ff-d4e68f1e5852 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:40.087949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:40.088567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:40.088904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:40.089178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:40.089932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:40.091458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:40.105073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:40.122596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bcd8b-bc2c-4d90-8e86-bb6e4a1d72df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:40.123230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bcd8b-bc2c-4d90-8e86-bb6e4a1d72df None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:40.123509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bcd8b-bc2c-4d90-8e86-bb6e4a1d72df None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:40.123779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bcd8b-bc2c-4d90-8e86-bb6e4a1d72df None admin] PATH_INFO: `/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:40.124911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bcd8b-bc2c-4d90-8e86-bb6e4a1d72df None admin] RBAC: Authorizing `identity:get_user(user_id=2afeb1cf31eb47f9a0c823030df9f4cb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:40.126387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bcd8b-bc2c-4d90-8e86-bb6e4a1d72df None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:40.127956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 20/40] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:39 2022] GET /identity/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb => generated 247 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:40.129220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8708bb-3057-4342-b11b-52caca2d9819 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:40.129504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8708bb-3057-4342-b11b-52caca2d9819 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:40.129856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8708bb-3057-4342-b11b-52caca2d9819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:40.147897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-df8708bb-3057-4342-b11b-52caca2d9819 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:40.149408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-df8708bb-3057-4342-b11b-52caca2d9819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:40.214069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 21/41] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:40.215104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8708bb-3057-4342-b11b-52caca2d9819 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:40.228771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 21/42] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Sep 1 13:40:39 2022] POST /identity/v3/roles => generated 223 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:40.339976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8708bb-3057-4342-b11b-52caca2d9819 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:40.355600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:40.378432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:40.379077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:40.379885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:40.380165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:40.381316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:40.382799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:40.384726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 22/43] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:40.390178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:40.456179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:40.488141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:40.498425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446161f-e271-4bb0-863d-990080d3c710 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:40.533684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:40.534398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:40.534677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:40.534959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] PATH_INFO: `/v3/domains/default/users/2afeb1cf31eb47f9a0c823030df9f4cb/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:40.537251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2afeb1cf31eb47f9a0c823030df9f4cb, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:40.538730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:40.589831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd2165-3240-4a68-936f-53d117bbc34c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:40.604025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 22/44] 10.222.0.61 () {58 vars in 1361 bytes} [Thu Sep 1 13:40:40 2022] PUT /identity/v3/domains/default/users/2afeb1cf31eb47f9a0c823030df9f4cb/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:40:40.625798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00572064-eb68-4d14-a5b2-50f89464ed97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:40.626488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00572064-eb68-4d14-a5b2-50f89464ed97 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:40.626771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00572064-eb68-4d14-a5b2-50f89464ed97 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:40.627047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00572064-eb68-4d14-a5b2-50f89464ed97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:40.628271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00572064-eb68-4d14-a5b2-50f89464ed97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:40.630407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00572064-eb68-4d14-a5b2-50f89464ed97 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:40.632187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 23/45] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:40.638156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:40.746724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeae2e9-38ff-4988-b77f-d5c2a76f94d3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:40.768188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:40.781504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a07ae-2e32-4bec-ba0b-b6892a1a2f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:40.782153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a07ae-2e32-4bec-ba0b-b6892a1a2f34 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:40.782435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a07ae-2e32-4bec-ba0b-b6892a1a2f34 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:40.782783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a07ae-2e32-4bec-ba0b-b6892a1a2f34 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:40.783367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a07ae-2e32-4bec-ba0b-b6892a1a2f34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:40.784812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a07ae-2e32-4bec-ba0b-b6892a1a2f34 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:40.887428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 23/46] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:40:40 2022] POST /identity/v3/projects => generated 312 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:42.496607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac3c9-e732-4fa5-9639-63882042f8e9 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:42.497161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac3c9-e732-4fa5-9639-63882042f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:42.497161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac3c9-e732-4fa5-9639-63882042f8e9 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:42.498521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 24/47] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:42.503828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d634db-15c9-4beb-bbb4-f723af0bbfc6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:42.504785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d634db-15c9-4beb-bbb4-f723af0bbfc6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:42.504896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d634db-15c9-4beb-bbb4-f723af0bbfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:42.523641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-82d634db-15c9-4beb-bbb4-f723af0bbfc6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:42.525157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-82d634db-15c9-4beb-bbb4-f723af0bbfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:42.591898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 24/48] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:42.794917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ee2c9-0bcd-480c-88f2-004d7fd5b9c7 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:42.795223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ee2c9-0bcd-480c-88f2-004d7fd5b9c7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:42.795504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ee2c9-0bcd-480c-88f2-004d7fd5b9c7 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:42.796867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 25/49] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:42.802467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddc13a-2652-4500-80ca-5e4cdea0849d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:42.802764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddc13a-2652-4500-80ca-5e4cdea0849d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:42.803046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddc13a-2652-4500-80ca-5e4cdea0849d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:42.821632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-aeddc13a-2652-4500-80ca-5e4cdea0849d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:42.822841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aeddc13a-2652-4500-80ca-5e4cdea0849d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:42.859994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ee2c9-0bcd-480c-88f2-004d7fd5b9c7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:42.890716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 25/50] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:42.915353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:42.966937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ee2c9-0bcd-480c-88f2-004d7fd5b9c7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:43.000302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:43.000937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:43.001223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:43.001502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:43.012170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:43.017466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:43.018451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:43.019268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 26/51] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:40:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:40:43.024348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddc13a-2652-4500-80ca-5e4cdea0849d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:43.058412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:43.127762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddc13a-2652-4500-80ca-5e4cdea0849d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:43.162615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:43.163234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:43.163516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:43.163858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:43.164534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:43.164840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:43.166284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:43.169317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:43.179866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 26/52] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:43.185438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:43.280240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45568ac-a4df-442c-8281-ef917b90d102 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:43.292447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:43.318315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:43.318964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:43.319248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:43.319524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:43.320674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:43.322193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:43.324322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 27/53] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:43.329524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:43.385120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:43.391683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837596-447d-4748-8ade-3e265dff4da0 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:43.427013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:43.427733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:43.428018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:43.428293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] PATH_INFO: `/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:43.429437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] RBAC: Authorizing `identity:get_user(user_id=2afeb1cf31eb47f9a0c823030df9f4cb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:43.429610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c052ea-feeb-4976-a1ea-4587d87ffcf3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:43.430954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:43.437243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 27/54] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:43 2022] GET /identity/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:43.438176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:43.463596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:43.464206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:43.464484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:43.464756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:43.465403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:43.466878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:43.539639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0abd1-6c55-4423-a448-fe8113b87228 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:43.573304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e2b6-bb8d-43df-829d-bf84c58ddaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:43.573954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e2b6-bb8d-43df-829d-bf84c58ddaa0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:43.574234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e2b6-bb8d-43df-829d-bf84c58ddaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:43.574502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e2b6-bb8d-43df-829d-bf84c58ddaa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:43.575623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658e2b6-bb8d-43df-829d-bf84c58ddaa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:43.577094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658e2b6-bb8d-43df-829d-bf84c58ddaa0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:43.577787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 28/55] 10.222.0.61 () {60 vars in 1140 bytes} [Thu Sep 1 13:40:43 2022] POST /identity/v3/projects => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:40:43.581472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 28/56] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:43.585358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:43.619434 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:43.695028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc80e44-25aa-4b68-9d56-37b83aac1706 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:43.732025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d507a8-9c25-4562-a633-68a517415460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:43.732646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d507a8-9c25-4562-a633-68a517415460 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:43.732928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d507a8-9c25-4562-a633-68a517415460 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:43.733229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d507a8-9c25-4562-a633-68a517415460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:43.733999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d507a8-9c25-4562-a633-68a517415460 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.domain.id=default, user.id=2afeb1cf31eb47f9a0c823030df9f4cb) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:43.734309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d507a8-9c25-4562-a633-68a517415460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:43.735788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d507a8-9c25-4562-a633-68a517415460 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:43.742570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 29/57] 10.222.0.61 () {56 vars in 1322 bytes} [Thu Sep 1 13:40:43 2022] GET /identity/v3/role_assignments?user.id=2afeb1cf31eb47f9a0c823030df9f4cb&scope.domain.id=default&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:45.491616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95a09a-4cc4-47a9-9b51-5f5bac9c8549 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:45.492186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95a09a-4cc4-47a9-9b51-5f5bac9c8549 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:45.492186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95a09a-4cc4-47a9-9b51-5f5bac9c8549 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:45.493821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 29/58] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:45.498797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27291f20-64f1-4637-b63d-976ec3aa6818 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:45.499180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27291f20-64f1-4637-b63d-976ec3aa6818 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:45.499347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27291f20-64f1-4637-b63d-976ec3aa6818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:45.517614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-27291f20-64f1-4637-b63d-976ec3aa6818 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:45.518827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-27291f20-64f1-4637-b63d-976ec3aa6818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:45.583922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 30/59] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:45.848017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95a09a-4cc4-47a9-9b51-5f5bac9c8549 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:45.878762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:45.948655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95a09a-4cc4-47a9-9b51-5f5bac9c8549 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:45.981392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001121c-5589-423c-a56c-53aa5ccc3295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:45.982031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001121c-5589-423c-a56c-53aa5ccc3295 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:45.982310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001121c-5589-423c-a56c-53aa5ccc3295 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:45.982578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001121c-5589-423c-a56c-53aa5ccc3295 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:45.983682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001121c-5589-423c-a56c-53aa5ccc3295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:45.985155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001121c-5589-423c-a56c-53aa5ccc3295 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:45.986806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 30/60] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:45.993171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27291f20-64f1-4637-b63d-976ec3aa6818 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:46.017562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:46.091543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27291f20-64f1-4637-b63d-976ec3aa6818 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:46.125427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd20f67-4a28-4aae-a45c-c9a59000ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:46.125984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd20f67-4a28-4aae-a45c-c9a59000ef82 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:46.126262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd20f67-4a28-4aae-a45c-c9a59000ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:46.126532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd20f67-4a28-4aae-a45c-c9a59000ef82 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:46.127168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd20f67-4a28-4aae-a45c-c9a59000ef82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:46.128622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd20f67-4a28-4aae-a45c-c9a59000ef82 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:46.244085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 31/61] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:40:45 2022] POST /identity/v3/users => generated 275 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:49.231311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc12bf-3fac-4cad-8c46-126ed72ef0c9 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.231815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc12bf-3fac-4cad-8c46-126ed72ef0c9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.231888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc12bf-3fac-4cad-8c46-126ed72ef0c9 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.233482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 31/62] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:49.239873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c20a0-7d34-4dd0-b319-9db8274139eb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.240791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c20a0-7d34-4dd0-b319-9db8274139eb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.241266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c20a0-7d34-4dd0-b319-9db8274139eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.270561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-527c20a0-7d34-4dd0-b319-9db8274139eb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:49.271794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-527c20a0-7d34-4dd0-b319-9db8274139eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:49.368288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 32/63] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:49.437373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e3836-9533-4522-81b6-2ef0b767fb4b None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.438059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e3836-9533-4522-81b6-2ef0b767fb4b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.438550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e3836-9533-4522-81b6-2ef0b767fb4b None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.440095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 32/64] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:49.445589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc009-7301-43ba-b30c-ea4cb3706526 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.445924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc009-7301-43ba-b30c-ea4cb3706526 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.446212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc009-7301-43ba-b30c-ea4cb3706526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.474987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5c6cc009-7301-43ba-b30c-ea4cb3706526 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:49.476205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5c6cc009-7301-43ba-b30c-ea4cb3706526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:49.482930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:49.577216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 33/65] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:49.693436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e3836-9533-4522-81b6-2ef0b767fb4b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:49.701539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24800-c569-4c78-84d0-76459355399c None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.701874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24800-c569-4c78-84d0-76459355399c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.702163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24800-c569-4c78-84d0-76459355399c None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.703777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 34/66] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:40:49.709984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6720-932f-4fca-b84d-7e99de746fdb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.710278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6720-932f-4fca-b84d-7e99de746fdb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.710563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6720-932f-4fca-b84d-7e99de746fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.730800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-7e5e6720-932f-4fca-b84d-7e99de746fdb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:49.732185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7e5e6720-932f-4fca-b84d-7e99de746fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:49.829715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:49.841795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 35/67] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:49.843709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e3836-9533-4522-81b6-2ef0b767fb4b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:49.853814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557228c9-a4f0-4371-bb38-071ebe4fd2fd None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.854181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557228c9-a4f0-4371-bb38-071ebe4fd2fd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.854537 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557228c9-a4f0-4371-bb38-071ebe4fd2fd None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.856204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 36/68] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:40:49.861799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c07c03-f29d-4970-9d86-4575d4a012cc None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.861938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c07c03-f29d-4970-9d86-4575d4a012cc None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.862323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c07c03-f29d-4970-9d86-4575d4a012cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.892095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c606-1c7f-4806-8f03-c299e47979d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:49.892743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c606-1c7f-4806-8f03-c299e47979d0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.893020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c606-1c7f-4806-8f03-c299e47979d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.893293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c606-1c7f-4806-8f03-c299e47979d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.896567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-96c07c03-f29d-4970-9d86-4575d4a012cc None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:49.898052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-96c07c03-f29d-4970-9d86-4575d4a012cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:49.905564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6c606-1c7f-4806-8f03-c299e47979d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-9db6c606-1c7f-4806-8f03-c299e47979d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:49.912655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:49.913491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:40:49.914296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 33/69] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:49.915030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e41a7-c37c-4dbc-ace5-fadce60158e1 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.915263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e41a7-c37c-4dbc-ace5-fadce60158e1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.916367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e41a7-c37c-4dbc-ace5-fadce60158e1 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.916956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 34/70] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:49.922036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42db149-2bcc-4138-a73d-bfa5f75bf3f2 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:49.922323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42db149-2bcc-4138-a73d-bfa5f75bf3f2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:49.922599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42db149-2bcc-4138-a73d-bfa5f75bf3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:49.944711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d42db149-2bcc-4138-a73d-bfa5f75bf3f2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:49.946789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d42db149-2bcc-4138-a73d-bfa5f75bf3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:49.977335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:49.983728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 37/71] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:49.984705 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c07c03-f29d-4970-9d86-4575d4a012cc None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.022738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 35/72] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:50.023615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42db149-2bcc-4138-a73d-bfa5f75bf3f2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.100544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c07c03-f29d-4970-9d86-4575d4a012cc None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.115682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.128571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42db149-2bcc-4138-a73d-bfa5f75bf3f2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.156906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.157564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.158041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.158327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.159015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:50.159325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.160762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.165685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.166363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.166647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.166926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.169797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 38/73] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:40:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:50.170700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.178276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:50.183278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:50.184219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:40:50.184990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 36/74] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:50.185600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.236130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.280860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e32fd-012e-4148-bde5-9ebc5d41c205 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.322944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cdad3-13de-4b9b-aee6-00c02ad2aee6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.325034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.326551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.326838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.327118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.342620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:50.348125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:50.349108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:50.350092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 39/75] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:50.354754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.362741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.363896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.364179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.364457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.380286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.381747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.383400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 37/76] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:50.384174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.393727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.458204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe27e4-a624-4493-b719-a14bd8106642 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.480130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.486983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4fe38-6303-44e2-ad5e-afe87c51baa9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.493347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.494071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.494359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.494642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.495312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:50.495619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.497013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.506189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 40/77] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:50.507079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.522514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.523238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.523519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.523772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.533529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:50.538786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:50.539723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:40:50.540517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 38/78] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:50.541164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.606679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179cf16-cb32-4ab1-a01a-3220fda0caed None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.607994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.641980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.642622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.642906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.643106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b29927-6796-4637-b6cd-d0c6545f6dc2 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.643212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.644356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.645855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.647424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 41/79] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:50.648286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.678871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.679484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.679768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.680042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.680716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:50.681088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.682506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.690768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 39/80] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:50.694675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.705147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.752697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423ff5-17ba-44c1-8d44-01736dea6e6b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.786096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.786695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.786980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.787255 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.794256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88ed70-c45a-408d-b99d-0dc8bc2ce411 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.821608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.829322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.830132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.830248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.830481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.831671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.833135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.834750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 40/81] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:50.835513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.854609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.856092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.912976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:50.944137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abfb43-7ed6-4249-858f-e2363560ad9f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:50.960964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 42/82] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:40:50.962672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:50.977889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:50.978509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:50.978792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:50.979066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:50.979738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] RBAC: Adding query filter params (name=member) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:50.980045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:50.981437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:50.989941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 41/83] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:50.989941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.049267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:51.065377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9d0a2-4424-4518-8738-a4740cd7d44e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:51.090833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5253af-0050-4bb8-a89b-ffb4077809bd None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:51.099909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:51.100526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:51.100808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:51.101083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:51.101730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:51.103215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:51.125015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:51.125648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:51.139245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:51.139245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] PATH_INFO: `/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:51.139245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] RBAC: Authorizing `identity:get_user(user_id=2afeb1cf31eb47f9a0c823030df9f4cb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:51.139245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:51.139245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 42/84] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:50 2022] GET /identity/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:51.139245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.228430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aab4c-4c0c-4a99-89b6-b139345dd346 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:51.239039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 43/85] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:40:50 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:51.246756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.254616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:51.257009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:51.267095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:51.267725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:51.268006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:51.268269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:51.357686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:51.359209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:51.377984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97884875-68e0-41d0-bc33-59f0dfd47b1d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:51.416705 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:51.417265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:51.417562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:51.417867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:51.419049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:51.421066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:51.422464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 44/86] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:51 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:51.423187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.492913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 43/87] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:40:51.493713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.523814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:51.531668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:51.555242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159566fa-e00d-4b01-a4b1-ac74d05c35dc None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:51.598263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:51.599133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:51.599565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:51.599837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:51.605098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:51.605098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:51.605098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 45/88] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:51 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:51.605914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.623163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f924da-f096-4488-9a98-94433612eb4c None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:51.671188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:51.671853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:51.672140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:51.672592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:51.730222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d4dfa-a632-438a-96c3-3f9135977d03 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:51.762205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:51.764598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:51.768211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:51.768825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:51.769108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:51.769411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:51.770236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=b4e7b7be35104fb680b3ffa073772913) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:51.770542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:51.772001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:51.777952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 46/89] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:40:51 2022] GET /identity/v3/role_assignments?user.id=b4e7b7be35104fb680b3ffa073772913&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:51.778845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.791560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:51.795689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:51.888103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 44/90] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:40:51.888984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:51.968543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84e5dd-bf48-4c8c-9d1e-efb3cb43cf4d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:52.008709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:52.009372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:52.009657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:52.009964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:52.011163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:52.012804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:52.014765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 47/91] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:51 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:52.015349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:52.100423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80847557-1540-4c89-a95a-be6932be6b70 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:52.126723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:52.155613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:52.169328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:52.169978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:52.170268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:52.170550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:52.180276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00260c-308b-45c8-975a-27899b745a2f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:52.247673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:52.248328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:52.248610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:52.248887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:52.250306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:52.251789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:52.255901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 48/92] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:52 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:52.256693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:52.306804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:52.308203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:52.425720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 45/93] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:40:52.426563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:52.458761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:52.512904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b26687-e884-4625-94fc-6510a30fc91c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:52.540325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017ac32-5981-459f-9e08-e082935d1a4e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:52.561982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:52.562531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:52.562818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:52.563100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:52.563919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] RBAC: Adding query filter params (role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=b4e7b7be35104fb680b3ffa073772913) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:52.564235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:52.565692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:52.572347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 49/94] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:40:52 2022] GET /identity/v3/role_assignments?user.id=b4e7b7be35104fb680b3ffa073772913&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:52.573438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:52.578569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:52.579254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:52.579549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:52.579828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:52.580588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=2afeb1cf31eb47f9a0c823030df9f4cb) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:52.580899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:52.582546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:52.591527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 46/95] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:40:52 2022] GET /identity/v3/role_assignments?user.id=2afeb1cf31eb47f9a0c823030df9f4cb&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:52.610270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:52.690667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a2f2d-5bac-4eb3-9053-9cb808ee5650 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:52.728525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2073786-fc90-4fec-87aa-6bbf7263c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:52.729205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2073786-fc90-4fec-87aa-6bbf7263c773 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:52.729490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2073786-fc90-4fec-87aa-6bbf7263c773 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:52.729796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2073786-fc90-4fec-87aa-6bbf7263c773 None admin] PATH_INFO: `/v3/projects/56175bd4364f48388ec476d9b5d757fc` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:52.731179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2073786-fc90-4fec-87aa-6bbf7263c773 None admin] RBAC: Authorizing `identity:get_project(project_id=56175bd4364f48388ec476d9b5d757fc)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:52.732745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2073786-fc90-4fec-87aa-6bbf7263c773 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:52.734610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 50/96] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:52 2022] GET /identity/v3/projects/56175bd4364f48388ec476d9b5d757fc => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:52.741064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:52.857463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:52.869542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dd98e-4569-42f4-83ed-793bd13fa646 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:52.920563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8901b-8624-4cbf-baac-db710393f5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:52.921200 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8901b-8624-4cbf-baac-db710393f5c1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:52.921483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8901b-8624-4cbf-baac-db710393f5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:52.921762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8901b-8624-4cbf-baac-db710393f5c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:52.922540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8901b-8624-4cbf-baac-db710393f5c1 None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=56175bd4364f48388ec476d9b5d757fc, user.id=b4e7b7be35104fb680b3ffa073772913) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:52.922851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8901b-8624-4cbf-baac-db710393f5c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:52.924321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8901b-8624-4cbf-baac-db710393f5c1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:52.931216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 47/97] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:40:52 2022] GET /identity/v3/role_assignments?user.id=b4e7b7be35104fb680b3ffa073772913&scope.project.id=56175bd4364f48388ec476d9b5d757fc&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:53.500719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82c171-1bc3-4ed7-a5af-09b58007330a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:53.501400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82c171-1bc3-4ed7-a5af-09b58007330a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:53.501972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82c171-1bc3-4ed7-a5af-09b58007330a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:53.503562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 51/98] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:53.509308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37464b-6102-476e-b57d-c1603fd1ce18 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:53.510189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37464b-6102-476e-b57d-c1603fd1ce18 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:53.510667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37464b-6102-476e-b57d-c1603fd1ce18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:53.534874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8a37464b-6102-476e-b57d-c1603fd1ce18 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:53.536937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8a37464b-6102-476e-b57d-c1603fd1ce18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:53.610982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 48/99] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:53.910826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82c171-1bc3-4ed7-a5af-09b58007330a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:53.955345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bd652-a847-4b48-b365-9bc2f95441c9 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:53.955781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bd652-a847-4b48-b365-9bc2f95441c9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:53.955970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bd652-a847-4b48-b365-9bc2f95441c9 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:53.958186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 49/100] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:53.964460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6a357-61bf-46e6-9898-2d73fdf3c48f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:53.964943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6a357-61bf-46e6-9898-2d73fdf3c48f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:53.965385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6a357-61bf-46e6-9898-2d73fdf3c48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:53.998837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4fd6a357-61bf-46e6-9898-2d73fdf3c48f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:54.000307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4fd6a357-61bf-46e6-9898-2d73fdf3c48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:54.054122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:54.099645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82c171-1bc3-4ed7-a5af-09b58007330a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:54.151314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 50/101] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:54.171730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:54.172304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:54.172589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:54.172866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:54.198414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:54.203913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:54.204741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:54.205505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 52/102] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:54.212046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6a357-61bf-46e6-9898-2d73fdf3c48f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:54.242062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:54.365167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6a357-61bf-46e6-9898-2d73fdf3c48f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:54.418495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:54.419129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:54.419412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:54.419688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:54.420369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:54.420738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:54.422206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:54.432137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 51/103] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:40:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:54.438348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:54.588792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b831-a99f-4518-9a7c-85781578bf57 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:54.635659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:54.636688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:54.637174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:54.637716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] PATH_INFO: `/v3/users/b47769234c3d4e1a83613623aa9346f6` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:54.654678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:54.660326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] RBAC: Authorizing `identity:get_user(user_id=b47769234c3d4e1a83613623aa9346f6)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:54.662096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:54.663943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 53/104] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:54 2022] GET /identity/v3/users/b47769234c3d4e1a83613623aa9346f6 => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:54.669598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:54.719902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:54.819927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:54.829898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8081c6-7622-443a-9c43-2deb5e1f6f8f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:54.873929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8bb27-3069-496a-b4b1-b775e54e4555 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:54.874407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045b21f-0589-4cf8-a574-5128ef2ec963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:54.875059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045b21f-0589-4cf8-a574-5128ef2ec963 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:54.875337 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045b21f-0589-4cf8-a574-5128ef2ec963 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:54.875628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045b21f-0589-4cf8-a574-5128ef2ec963 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:54.882457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045b21f-0589-4cf8-a574-5128ef2ec963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:54.883923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045b21f-0589-4cf8-a574-5128ef2ec963 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:54.885522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 54/105] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:40:54.886689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e939f9-1749-419e-9cab-ec5b8a2c10fd None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:54.886967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e939f9-1749-419e-9cab-ec5b8a2c10fd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:54.887236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e939f9-1749-419e-9cab-ec5b8a2c10fd None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:54.888435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 55/106] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:54.892885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e939f9-1749-419e-9cab-ec5b8a2c10fd None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:54.939651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae260c-eb5c-4453-8a5c-d0e269c11c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:54.941882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae260c-eb5c-4453-8a5c-d0e269c11c42 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:54.942299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae260c-eb5c-4453-8a5c-d0e269c11c42 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:54.942714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae260c-eb5c-4453-8a5c-d0e269c11c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.014252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:55.047999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae260c-eb5c-4453-8a5c-d0e269c11c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:55.049534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae260c-eb5c-4453-8a5c-d0e269c11c42 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:55.114569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e939f9-1749-419e-9cab-ec5b8a2c10fd None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:55.169550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:55.170213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.170501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.170780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.171448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:55.172911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:55.224467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:55.244124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 52/107] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:40:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:40:55.245465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103318-cf87-4c0c-8092-9e13019eabf8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.245834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103318-cf87-4c0c-8092-9e13019eabf8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.250229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e103318-cf87-4c0c-8092-9e13019eabf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.272052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0e103318-cf87-4c0c-8092-9e13019eabf8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:55.273267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0e103318-cf87-4c0c-8092-9e13019eabf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:55.297525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 56/108] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:40:54 2022] POST /identity/v3/projects => generated 302 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:40:55.298276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:55.416284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 53/109] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:55.479181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed21d36-4352-4e5a-b861-ce804a862392 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:55.503985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:55.526546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c210e-1092-4842-822d-1fadc169df6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:55.527119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c210e-1092-4842-822d-1fadc169df6f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.527400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c210e-1092-4842-822d-1fadc169df6f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.527679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c210e-1092-4842-822d-1fadc169df6f None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.528998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c210e-1092-4842-822d-1fadc169df6f None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:55.530555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c210e-1092-4842-822d-1fadc169df6f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:55.532459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 57/110] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:55 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:55.538822 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e103318-cf87-4c0c-8092-9e13019eabf8 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:55.574067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899716bd-a5f3-4a85-a9ff-fa937ad9d18b None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.574427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899716bd-a5f3-4a85-a9ff-fa937ad9d18b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.574641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899716bd-a5f3-4a85-a9ff-fa937ad9d18b None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.576388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 58/111] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:40:55.581377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd16c55-8c78-4ff1-b16f-7d699ef18f22 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.582528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd16c55-8c78-4ff1-b16f-7d699ef18f22 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.583029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd16c55-8c78-4ff1-b16f-7d699ef18f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.603810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5bd16c55-8c78-4ff1-b16f-7d699ef18f22 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:55.605318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5bd16c55-8c78-4ff1-b16f-7d699ef18f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:55.687975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 59/112] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:55.690437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244e056-5e55-4969-bb2d-7dde0e4aa13a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.692065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244e056-5e55-4969-bb2d-7dde0e4aa13a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.693296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244e056-5e55-4969-bb2d-7dde0e4aa13a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.694835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 60/113] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:55.700659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e103318-cf87-4c0c-8092-9e13019eabf8 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:55.704523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf3e42-97fd-4caa-a518-0a480a536ad9 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.705011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf3e42-97fd-4caa-a518-0a480a536ad9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.705496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf3e42-97fd-4caa-a518-0a480a536ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.718958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:55.738713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-75cf3e42-97fd-4caa-a518-0a480a536ad9 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:55.745231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-75cf3e42-97fd-4caa-a518-0a480a536ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:55.764201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ccffb-cd06-4e38-9a0e-c0d70fb01df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:55.765374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ccffb-cd06-4e38-9a0e-c0d70fb01df1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.766172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ccffb-cd06-4e38-9a0e-c0d70fb01df1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.766655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ccffb-cd06-4e38-9a0e-c0d70fb01df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.767584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ccffb-cd06-4e38-9a0e-c0d70fb01df1 None admin] RBAC: Adding query filter params (role.id=89ffb3825da1475c8ae689ccea2455ea, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=b47769234c3d4e1a83613623aa9346f6) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:55.768087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ccffb-cd06-4e38-9a0e-c0d70fb01df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:55.771929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ccffb-cd06-4e38-9a0e-c0d70fb01df1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:55.797898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 54/114] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:40:55 2022] GET /identity/v3/role_assignments?user.id=b47769234c3d4e1a83613623aa9346f6&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=89ffb3825da1475c8ae689ccea2455ea => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:55.797898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec54317-5b2a-45da-ae9e-0c661f723b54 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.797898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec54317-5b2a-45da-ae9e-0c661f723b54 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.797898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec54317-5b2a-45da-ae9e-0c661f723b54 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.797898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 55/115] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:55.797898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbe98b-dc78-4de5-b103-6a660be2548a None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:55.798382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbe98b-dc78-4de5-b103-6a660be2548a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:55.798382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbe98b-dc78-4de5-b103-6a660be2548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:55.819510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-00bbe98b-dc78-4de5-b103-6a660be2548a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:55.821461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-00bbe98b-dc78-4de5-b103-6a660be2548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:55.852877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 61/116] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:55.853565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf3e42-97fd-4caa-a518-0a480a536ad9 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:55.906195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:55.936158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 56/117] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:40:55.992035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf3e42-97fd-4caa-a518-0a480a536ad9 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:56.049456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:56.050091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:56.050344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:56.050616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:56.062644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:56.068324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:56.069215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:56.070447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 62/118] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:56.075490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbe98b-dc78-4de5-b103-6a660be2548a None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:56.159253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:56.217144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:56.222224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbe98b-dc78-4de5-b103-6a660be2548a None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:56.309430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:56.310180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:56.311171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:56.311171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:56.311458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] RBAC: Adding query filter params (name=member) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:56.311704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:56.323563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:56.348052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 57/119] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:40:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:56.348932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:56.405240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3e073-160a-45b1-aa04-09988b80ac96 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:56.406819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:56.448124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:56.449095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:56.449579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:56.452738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:56.466881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:56.472496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:56.473597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:40:56.474630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:56.476696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 63/120] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:40:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:40:56.477730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:56.598243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0a0ba-2203-4419-be74-35cc8ac680da None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:56.636871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:56.637959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:56.638471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:56.638960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:56.650422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:56.656500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:56.657375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:40:56.658497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:40:56.661689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 58/121] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:56.662926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:56.679936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e8bb8-f32b-4397-9e39-69a401456095 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:56.691211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:56.768441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:56.769117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:56.769510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:56.769948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:56.771917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:56.773431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:56.779467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 64/122] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:56 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:56.784183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:56.786793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa3c9-f054-4958-947f-9dbba88bc737 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:56.833475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:56.834512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:56.835043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:56.835456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:56.847563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:56.861315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:56.862273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:40:56.863093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:40:56.870316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:56.870914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 59/123] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:40:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:40:56.872032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:56.947796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876eab-f20c-4391-83e4-1a8976aa0189 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:56.979238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:56.986916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:56.987549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:56.988422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:56.988699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:56.989384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:56.989689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:56.991149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.005613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 65/124] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:40:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.005613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.009095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657623b-8867-4a37-9bea-5d7227074003 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.047136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.048057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.048492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.049038 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.049890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] RBAC: Adding query filter params (name=member) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:57.050916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.052489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.062066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 60/125] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:40:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.063279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.113321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:57.124504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd7eab-9f2f-4b44-978f-3bdb8b6f5c41 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.163359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.164011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.164341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.164546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.165676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.167199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.168877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 66/126] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.173781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.182543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9100474-4727-4c4a-b49d-7e687c994cce None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.219093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.220030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.220539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.221011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.221878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:57.222915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.224494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.233453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 61/127] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.239047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.248934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:57.286349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3e91-0d1e-4cb2-8f40-017dacf7d0fe None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.323929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.324393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.324677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.324995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.326190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.327942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.334612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 67/128] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.334612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.374372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:57.376608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e49f7f-cfd2-4637-8eba-190cc42c4d75 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.423160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.423823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.424101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.424482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.425997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.427982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.434260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 62/129] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.434260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.466430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d9f6c-2a93-46a2-908c-da5ead214bb4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.501582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.502283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.502562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.502932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/b4e7b7be35104fb680b3ffa073772913/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.505575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] RBAC: Authorizing `identity:create_grant(project_id=bca7c40ebb9145f49bb3f6af4bb61457, user_id=b4e7b7be35104fb680b3ffa073772913, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.507141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.524287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:57.548962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c2bf8-8500-47b4-b811-bc5e1f52fe6d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.587543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.588184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.588469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.588743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] PATH_INFO: `/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.588918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 68/130] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:40:57 2022] PUT /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/b4e7b7be35104fb680b3ffa073772913/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:40:57.590196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.599454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] RBAC: Authorizing `identity:get_user(user_id=2afeb1cf31eb47f9a0c823030df9f4cb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.601627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.604028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 63/131] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.606463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.733334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77e614-194e-43a7-8c80-e8c8d15720fa None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.775851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.776476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:57.777119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.778093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.778692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] PATH_INFO: `/v3/projects/56175bd4364f48388ec476d9b5d757fc` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.784239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] RBAC: Authorizing `identity:get_project(project_id=56175bd4364f48388ec476d9b5d757fc)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.787872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.790508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 64/132] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/projects/56175bd4364f48388ec476d9b5d757fc => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.794576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.818137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e87a2-2421-4742-b544-cb6a32e048ba None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.835983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:57.872470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.873112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.873392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.873664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.874875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:57.876366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:57.878070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 69/133] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:57.879476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:57.954186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec8f0d-6c2f-4272-9fb7-78c5ad09c291 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:57.996376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4088f-3271-4b1e-9aff-255dfc098a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:57.997266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4088f-3271-4b1e-9aff-255dfc098a2e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:57.997846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4088f-3271-4b1e-9aff-255dfc098a2e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:57.998131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4088f-3271-4b1e-9aff-255dfc098a2e None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:57.999390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4088f-3271-4b1e-9aff-255dfc098a2e None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:58.001152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4088f-3271-4b1e-9aff-255dfc098a2e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:58.002493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fe6a8-3296-4272-b28e-e0b139889d6c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:58.003016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 65/134] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:57 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:58.004663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8840c-3dee-4476-a2d1-7229bcef082f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.004954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8840c-3dee-4476-a2d1-7229bcef082f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.005231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8840c-3dee-4476-a2d1-7229bcef082f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.010632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 66/135] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:58.010632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8840c-3dee-4476-a2d1-7229bcef082f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:58.047716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acfaa3-c57a-44e2-87ec-cbe56a1a6b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:58.048441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acfaa3-c57a-44e2-87ec-cbe56a1a6b41 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.048723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acfaa3-c57a-44e2-87ec-cbe56a1a6b41 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.049001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acfaa3-c57a-44e2-87ec-cbe56a1a6b41 None admin] PATH_INFO: `/v3/projects/56175bd4364f48388ec476d9b5d757fc/users/b4e7b7be35104fb680b3ffa073772913/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.051648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acfaa3-c57a-44e2-87ec-cbe56a1a6b41 None admin] RBAC: Authorizing `identity:create_grant(project_id=56175bd4364f48388ec476d9b5d757fc, user_id=b4e7b7be35104fb680b3ffa073772913, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:58.053237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acfaa3-c57a-44e2-87ec-cbe56a1a6b41 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:58.090482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:58.112152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8840c-3dee-4476-a2d1-7229bcef082f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:58.146478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30dcff-ee03-4efe-8d4f-d9930c08eade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:58.147135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30dcff-ee03-4efe-8d4f-d9930c08eade None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.147418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30dcff-ee03-4efe-8d4f-d9930c08eade None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.147694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30dcff-ee03-4efe-8d4f-d9930c08eade None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/b4e7b7be35104fb680b3ffa073772913/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.149880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30dcff-ee03-4efe-8d4f-d9930c08eade None admin] RBAC: Authorizing `identity:create_grant(project_id=bca7c40ebb9145f49bb3f6af4bb61457, user_id=b4e7b7be35104fb680b3ffa073772913, role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:58.151333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30dcff-ee03-4efe-8d4f-d9930c08eade None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:58.159185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 70/136] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:40:57 2022] PUT /identity/v3/projects/56175bd4364f48388ec476d9b5d757fc/users/b4e7b7be35104fb680b3ffa073772913/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:40:58.170433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba1d8-98f0-48df-bd37-13ed8ebd1b47 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.170433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba1d8-98f0-48df-bd37-13ed8ebd1b47 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.170433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba1d8-98f0-48df-bd37-13ed8ebd1b47 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.170433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 71/137] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:40:58.170433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ba1d8-98f0-48df-bd37-13ed8ebd1b47 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:58.194840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:58.246922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 67/138] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:40:58 2022] PUT /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/b4e7b7be35104fb680b3ffa073772913/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:40:58.248501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e609-e67e-4e5c-adec-e7b2eddd9ccb None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.248892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e609-e67e-4e5c-adec-e7b2eddd9ccb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.250206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e609-e67e-4e5c-adec-e7b2eddd9ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.271739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f434e609-e67e-4e5c-adec-e7b2eddd9ccb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:58.273209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f434e609-e67e-4e5c-adec-e7b2eddd9ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:58.307225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ba1d8-98f0-48df-bd37-13ed8ebd1b47 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:58.384665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 68/139] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:58.386713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c983ae-f3d2-4ec0-a7a8-54ff1f34768a None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.387005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c983ae-f3d2-4ec0-a7a8-54ff1f34768a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.387280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c983ae-f3d2-4ec0-a7a8-54ff1f34768a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.393470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:58.394229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.394509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.394781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/2afeb1cf31eb47f9a0c823030df9f4cb/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.397439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] RBAC: Authorizing `identity:create_grant(project_id=bca7c40ebb9145f49bb3f6af4bb61457, user_id=2afeb1cf31eb47f9a0c823030df9f4cb, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:58.398956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:58.421663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c0c983ae-f3d2-4ec0-a7a8-54ff1f34768a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:40:58.422917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c0c983ae-f3d2-4ec0-a7a8-54ff1f34768a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:40:58.498503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 72/140] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:40:58 2022] PUT /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/2afeb1cf31eb47f9a0c823030df9f4cb/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:40:58.506759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:58.509868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 69/141] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:40:58.723996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:58.822740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:58.851990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c7a96-c45c-4440-9a49-3e52032f4148 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:58.900226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:58.900886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:58.901168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:58.901442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:58.902718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:58.904232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:58.907722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 73/142] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:40:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:58.914423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c983ae-f3d2-4ec0-a7a8-54ff1f34768a None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:59.046931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c983ae-f3d2-4ec0-a7a8-54ff1f34768a None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:59.069681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:59.112310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:59.112976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:59.113416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:59.113697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:59.114407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:59.115881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:59.233597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a19be7-4408-4b88-9e74-235f95e7df0a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:59.258856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 70/143] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:40:58 2022] POST /identity/v3/users => generated 283 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:40:59.292789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:59.293508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:59.293827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:59.294110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:59.322509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:40:59.328280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:40:59.330413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:40:59.332306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 74/144] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:40:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:40:59.337906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:59.435267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:59.493552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc900e-2709-492a-8acf-85be7260e623 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:59.545215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:59.545878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:59.546159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:59.546436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:59.547157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:40:59.547460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:59.548932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:59.558681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 71/145] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:40:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:59.564273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:59.572997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:59.723849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7818171-349b-40cf-bb2c-8ef1d6cd3883 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:59.770739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:59.771735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:59.772381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:59.772856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] PATH_INFO: `/v3/users/b47769234c3d4e1a83613623aa9346f6` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:59.775374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] RBAC: Authorizing `identity:get_user(user_id=b47769234c3d4e1a83613623aa9346f6)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:59.777073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:59.778980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 75/146] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:40:59 2022] GET /identity/v3/users/b47769234c3d4e1a83613623aa9346f6 => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:59.786972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:40:59.904999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023a1d7-9a3e-4c2e-977b-5dfa32f199e7 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:40:59.922527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:40:59.944537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f6834-1064-4c66-b93b-09daf723df80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:40:59.945181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6834-1064-4c66-b93b-09daf723df80 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:40:59.945464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6834-1064-4c66-b93b-09daf723df80 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:40:59.945833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f6834-1064-4c66-b93b-09daf723df80 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:40:59.946924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f6834-1064-4c66-b93b-09daf723df80 None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:40:59.948351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f6834-1064-4c66-b93b-09daf723df80 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:40:59.950013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 72/147] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:40:59 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:40:59.955872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:00.001666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c8b4-0089-4497-a24e-40b281fa0970 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:00.001891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c8b4-0089-4497-a24e-40b281fa0970 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:00.002192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c8b4-0089-4497-a24e-40b281fa0970 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:00.005312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 73/148] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:00.009448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9eae4-efdf-47ea-9683-9b93bd8a5dd5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:00.009647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9eae4-efdf-47ea-9683-9b93bd8a5dd5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:00.009969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9eae4-efdf-47ea-9683-9b93bd8a5dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:00.040975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-afc9eae4-efdf-47ea-9683-9b93bd8a5dd5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:00.042769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-afc9eae4-efdf-47ea-9683-9b93bd8a5dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:00.061333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:00.111419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fa1a9-388c-471e-ab38-6b541d20312d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:00.174604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:00.175810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:00.176086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:00.176356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/b47769234c3d4e1a83613623aa9346f6/roles/89ffb3825da1475c8ae689ccea2455ea` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:00.178815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] RBAC: Authorizing `identity:create_grant(project_id=bca7c40ebb9145f49bb3f6af4bb61457, user_id=b47769234c3d4e1a83613623aa9346f6, role_id=89ffb3825da1475c8ae689ccea2455ea)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:00.181392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:00.204478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 74/149] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:00.280358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 76/150] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:40:59 2022] PUT /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/users/b47769234c3d4e1a83613623aa9346f6/roles/89ffb3825da1475c8ae689ccea2455ea => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:00.599972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9eae4-efdf-47ea-9683-9b93bd8a5dd5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:00.715006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9eae4-efdf-47ea-9683-9b93bd8a5dd5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:00.730441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:00.756001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:00.756657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:00.756938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:00.757206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:00.769865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:00.774772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:00.775607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:00.776799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 75/151] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:00.782059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:00.901831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16adc99-b96f-482d-aa91-3b73efddf2fb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:00.932886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:00.937857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f385e0-6731-4c08-8aba-32e2cfbd597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:00.938535 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f385e0-6731-4c08-8aba-32e2cfbd597c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:00.938814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f385e0-6731-4c08-8aba-32e2cfbd597c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:00.939085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f385e0-6731-4c08-8aba-32e2cfbd597c None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:00.939763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f385e0-6731-4c08-8aba-32e2cfbd597c None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:00.940062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f385e0-6731-4c08-8aba-32e2cfbd597c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:00.942100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f385e0-6731-4c08-8aba-32e2cfbd597c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:00.951544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 77/152] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:00.957213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:00.969285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abadd7-07e4-4e28-b16b-310b1073f592 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:00.969581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abadd7-07e4-4e28-b16b-310b1073f592 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:00.969857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abadd7-07e4-4e28-b16b-310b1073f592 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:00.973392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 78/153] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:00.979563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc88c-c82f-4f57-b832-f0686edc0ecb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:00.979845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc88c-c82f-4f57-b832-f0686edc0ecb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:00.980116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc88c-c82f-4f57-b832-f0686edc0ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:01.013617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-586cc88c-c82f-4f57-b832-f0686edc0ecb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:01.014869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-586cc88c-c82f-4f57-b832-f0686edc0ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:01.131577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 79/154] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:01.149966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4b078-351a-49d6-bd5d-cb3fe67315bb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:01.193444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:01.194034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:01.194311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:01.194581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:01.198532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:01.200073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:01.202017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 76/155] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:00 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:01.207819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cc88c-c82f-4f57-b832-f0686edc0ecb None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:01.342709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cc88c-c82f-4f57-b832-f0686edc0ecb None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:01.369903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:01.387038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795ceba-50a6-4f4d-a9f1-8b5573ab3ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:01.387674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795ceba-50a6-4f4d-a9f1-8b5573ab3ac1 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:01.387953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795ceba-50a6-4f4d-a9f1-8b5573ab3ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:01.388227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795ceba-50a6-4f4d-a9f1-8b5573ab3ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:01.469317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795ceba-50a6-4f4d-a9f1-8b5573ab3ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:01.470860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795ceba-50a6-4f4d-a9f1-8b5573ab3ac1 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:01.590490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 80/156] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:01.595731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:01.674602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:01.730254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc63ae-460c-4796-86dc-ae4aed9034a9 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:01.730529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc63ae-460c-4796-86dc-ae4aed9034a9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:01.730730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc63ae-460c-4796-86dc-ae4aed9034a9 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:01.732252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 81/157] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:01.738679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bea986-e617-4300-9769-21b76d4ccb14 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:01.738968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bea986-e617-4300-9769-21b76d4ccb14 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:01.739235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bea986-e617-4300-9769-21b76d4ccb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:01.744641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977ea3-729e-4434-af8b-22feb57a92c8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:01.770957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-42bea986-e617-4300-9769-21b76d4ccb14 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:01.772182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-42bea986-e617-4300-9769-21b76d4ccb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:01.814157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:01.818550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:01.819210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:01.819490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:01.819762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:01.822266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:01.823870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:01.830928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 77/158] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:01 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:01.832486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:01.892424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 82/159] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:01.893386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bea986-e617-4300-9769-21b76d4ccb14 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:02.042224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada943aa-c0fe-4ee0-a587-9ccd0a0e9a34 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:02.065552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:02.068534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bea986-e617-4300-9769-21b76d4ccb14 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:02.130757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb704341-1d0f-4d6b-8883-57810602684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:02.131376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb704341-1d0f-4d6b-8883-57810602684c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.131659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb704341-1d0f-4d6b-8883-57810602684c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.131983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb704341-1d0f-4d6b-8883-57810602684c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.132786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb704341-1d0f-4d6b-8883-57810602684c None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=b4e7b7be35104fb680b3ffa073772913) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:02.133095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb704341-1d0f-4d6b-8883-57810602684c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:02.134557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb704341-1d0f-4d6b-8883-57810602684c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:02.147013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 78/160] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:01 2022] GET /identity/v3/role_assignments?user.id=b4e7b7be35104fb680b3ffa073772913&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:02.148209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ecfe8-d506-44f5-b433-2b72eebc9420 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.148570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ecfe8-d506-44f5-b433-2b72eebc9420 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.148963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ecfe8-d506-44f5-b433-2b72eebc9420 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.150242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 79/161] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:02.155813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984eafe-f151-402e-8f83-41f53ab2ed4c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.156140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984eafe-f151-402e-8f83-41f53ab2ed4c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.156417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984eafe-f151-402e-8f83-41f53ab2ed4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.160155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:02.160812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.161085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.161350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.182452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:02.187573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:02.188567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:02.189737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:02.190356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-6984eafe-f151-402e-8f83-41f53ab2ed4c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:02.190356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6984eafe-f151-402e-8f83-41f53ab2ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:02.190490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 83/162] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:02.197950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:02.236505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:02.351688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 80/163] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:02.398686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de777967-9518-4031-a58d-fe1b9478991a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:02.458078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:02.458749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.459034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.459309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.459980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:02.460287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:02.461688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:02.476783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 84/164] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:02.482689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984eafe-f151-402e-8f83-41f53ab2ed4c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:02.493804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:02.627250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984eafe-f151-402e-8f83-41f53ab2ed4c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:02.630608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:02.634123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:02.678092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:02.679134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.679640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.680148 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.681560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:02.683293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:02.685427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 81/165] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:02 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:02.692937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443fd3d-3591-44fa-8c59-de4da271aabf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:02.701860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:02.755892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:02.756553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.756834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.757111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.782730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:02.788868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:02.789835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:02.790929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:02.793144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 85/166] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:02.799111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:02.843397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6e6b6-fbd8-4fa6-8c5f-e7d739d96dc3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:02.893431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:02.894597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:02.894911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:02.895185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:02.967783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:02.980293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98f15f-a299-41b8-af9b-480e28b7601b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:03.000286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:03.001802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:03.033739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:03.034472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.034757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.035031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.035708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:03.036011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:03.037408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:03.050636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:03.054982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 86/167] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:03.055814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:03.204736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 82/168] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:03.205528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:03.226223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2d810-abae-4f75-b4bb-8d66304ea132 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:03.293657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:03.294332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.294615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.294892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.322180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:03.336077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:03.337172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:03.338178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:03.340208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 87/169] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:03.341200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:03.348396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:03.386016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ee05d-2703-48ea-99c6-45b0e74fc025 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:03.438471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:03.439124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.439354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:03.439800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.439800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] PATH_INFO: `/v3/users/b4e7b7be35104fb680b3ffa073772913` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.440862 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] RBAC: Authorizing `identity:get_user(user_id=b4e7b7be35104fb680b3ffa073772913)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:03.442380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:03.444378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 83/170] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/users/b4e7b7be35104fb680b3ffa073772913 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:03.445238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:03.491809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4d3e-2feb-4d86-81b8-50cae3f70037 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:03.559724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:03.560366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.560649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.560924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] PATH_INFO: `/v3/projects/56175bd4364f48388ec476d9b5d757fc` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.562287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] RBAC: Authorizing `identity:get_project(project_id=56175bd4364f48388ec476d9b5d757fc)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:03.563758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:03.565635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 88/171] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/projects/56175bd4364f48388ec476d9b5d757fc => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:03.566247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:03.641651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75409451-64d5-4f38-ad33-7c94d741a6bc None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:03.671293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b8cb-93d0-454d-9eaa-f5647ddd3124 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:03.696801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:03.698128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:03.698373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.699479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.700019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.700915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:03.701419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:03.703025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:03.706778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:03.707394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.707675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.707952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.722404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 84/172] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:03.723665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:03.738050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:03.786088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:03.789793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:03.891380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcb483-302f-4001-9920-23690d0bffdc None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:03.892170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 89/173] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:03.893028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:03.921678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:03.931790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6721-729f-41c1-bd33-d936b2a00568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:03.932680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6721-729f-41c1-bd33-d936b2a00568 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.934048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6721-729f-41c1-bd33-d936b2a00568 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.934048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6721-729f-41c1-bd33-d936b2a00568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.935509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad6721-729f-41c1-bd33-d936b2a00568 None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=56175bd4364f48388ec476d9b5d757fc, user.id=b4e7b7be35104fb680b3ffa073772913) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:03.936048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad6721-729f-41c1-bd33-d936b2a00568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:03.937697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad6721-729f-41c1-bd33-d936b2a00568 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:03.944286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 85/174] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/role_assignments?user.id=b4e7b7be35104fb680b3ffa073772913&scope.project.id=56175bd4364f48388ec476d9b5d757fc&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:03.945988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a397-e744-4bcc-b1c8-24a56e5ca195 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.946858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a397-e744-4bcc-b1c8-24a56e5ca195 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.947620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a397-e744-4bcc-b1c8-24a56e5ca195 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.950266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 86/175] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:03.952691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9554db-85a3-440f-ab1a-af536052cd91 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:03.953249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9554db-85a3-440f-ab1a-af536052cd91 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:03.953727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9554db-85a3-440f-ab1a-af536052cd91 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:03.957275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 87/176] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:03.958463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9554db-85a3-440f-ab1a-af536052cd91 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:04.030810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ccf49-ee41-4c72-9d41-4a23fc74d0e7 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:04.070906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143472b-a4ab-42a4-bc23-74194416f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:04.071538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143472b-a4ab-42a4-bc23-74194416f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.071888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143472b-a4ab-42a4-bc23-74194416f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.072166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143472b-a4ab-42a4-bc23-74194416f4c4 None admin] PATH_INFO: `/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.073400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143472b-a4ab-42a4-bc23-74194416f4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=2afeb1cf31eb47f9a0c823030df9f4cb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:04.074921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143472b-a4ab-42a4-bc23-74194416f4c4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:04.076554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 90/177] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/users/2afeb1cf31eb47f9a0c823030df9f4cb => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:04.077913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9c36-87f4-4622-83da-fd4108812180 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.078137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9c36-87f4-4622-83da-fd4108812180 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.078583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9c36-87f4-4622-83da-fd4108812180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.103874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4adb9c36-87f4-4622-83da-fd4108812180 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:04.105095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4adb9c36-87f4-4622-83da-fd4108812180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:04.141513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:04.179283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 91/178] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:04.180671 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0826a5a-bfea-4830-9ef7-6b4cd9d78ad4 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.180971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0826a5a-bfea-4830-9ef7-6b4cd9d78ad4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.181241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0826a5a-bfea-4830-9ef7-6b4cd9d78ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.200442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d0826a5a-bfea-4830-9ef7-6b4cd9d78ad4 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:04.201648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d0826a5a-bfea-4830-9ef7-6b4cd9d78ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:04.224171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:04.246493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9554db-85a3-440f-ab1a-af536052cd91 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:04.281094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 92/179] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:04.281905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0826a5a-bfea-4830-9ef7-6b4cd9d78ad4 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:04.327131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:04.327707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.327985 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.328256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.334508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:04.336050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:04.337738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 88/180] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:03 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:04.350770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:04.425933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0826a5a-bfea-4830-9ef7-6b4cd9d78ad4 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:04.466816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f74d9-b688-4552-81a1-1cbfc130e0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:04.467428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f74d9-b688-4552-81a1-1cbfc130e0d2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.467711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f74d9-b688-4552-81a1-1cbfc130e0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.467984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f74d9-b688-4552-81a1-1cbfc130e0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.495669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:04.518858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c03982-b798-481c-aab3-9c887a7dfe4d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:04.546442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f74d9-b688-4552-81a1-1cbfc130e0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:04.548664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f74d9-b688-4552-81a1-1cbfc130e0d2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:04.573118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fdcd1-eb34-4afe-9966-ce7ba7be7c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:04.573729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fdcd1-eb34-4afe-9966-ce7ba7be7c0f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.574031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fdcd1-eb34-4afe-9966-ce7ba7be7c0f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.574303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fdcd1-eb34-4afe-9966-ce7ba7be7c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.575043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fdcd1-eb34-4afe-9966-ce7ba7be7c0f None admin] RBAC: Adding query filter params (role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=b4e7b7be35104fb680b3ffa073772913) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:04.575352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fdcd1-eb34-4afe-9966-ce7ba7be7c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:04.576799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fdcd1-eb34-4afe-9966-ce7ba7be7c0f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:04.578758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:04.589796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 89/181] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:04 2022] GET /identity/v3/role_assignments?user.id=b4e7b7be35104fb680b3ffa073772913&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:04.589796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c44dc9-b8eb-4985-851d-55ef13502e48 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.589796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c44dc9-b8eb-4985-851d-55ef13502e48 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.589796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c44dc9-b8eb-4985-851d-55ef13502e48 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.589796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 90/182] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:04.589796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f144f3-c273-4379-8e7b-a8f93bb77558 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.590584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f144f3-c273-4379-8e7b-a8f93bb77558 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.590584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f144f3-c273-4379-8e7b-a8f93bb77558 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.591104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 91/183] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:04.593464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d34b3d-8b79-442c-98da-0a768b89d4e5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.593937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d34b3d-8b79-442c-98da-0a768b89d4e5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.594286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d34b3d-8b79-442c-98da-0a768b89d4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.635058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-58d34b3d-8b79-442c-98da-0a768b89d4e5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:04.636457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-58d34b3d-8b79-442c-98da-0a768b89d4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:04.692730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 93/184] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:04.694330 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f957e2-db6d-4313-8b0f-37dec8902ce0 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.694759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f957e2-db6d-4313-8b0f-37dec8902ce0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.695162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f957e2-db6d-4313-8b0f-37dec8902ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.714994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-38f957e2-db6d-4313-8b0f-37dec8902ce0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:04.716219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-38f957e2-db6d-4313-8b0f-37dec8902ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:04.733949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 92/185] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:04.735306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d34b3d-8b79-442c-98da-0a768b89d4e5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:04.786104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:04.809805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 94/186] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:04.809805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f957e2-db6d-4313-8b0f-37dec8902ce0 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:04.858214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d34b3d-8b79-442c-98da-0a768b89d4e5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:04.895263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:04.905654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:04.906460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.906741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.933581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.933581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:04.933787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f957e2-db6d-4313-8b0f-37dec8902ce0 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:04.933866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:04.934700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 93/187] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:04.935484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:04.971909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:04.972544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:04.972821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:04.973143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:04.988592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:04.995799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:04.996997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:04.997992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:05.000514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 95/188] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:05.001908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.060873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107afc7c-1680-424d-a2cf-5b8a508d5bab None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.063846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.098571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.099495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.099875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.100147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.101382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.102881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.104602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 94/189] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:04 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.105366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.123207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b4979-ab88-4d2b-aa30-408962b93ac5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.156711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.157356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.157632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.157952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.158618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:05.158948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.160882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.165009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.169253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 96/190] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.170111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.209197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c715f-59f4-4290-904f-f12b41c5665e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.245226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.245860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.246139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.246411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.247082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:05.247600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.248746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.257721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 95/191] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.258537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.274806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796220cb-dc31-4d1b-bceb-5e8327cf2117 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.282508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.311108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.311790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.312075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.312350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.323213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:05.328160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:05.329106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:05.329966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 97/192] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:05.330606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.368014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30210a2-60ec-48b3-b538-5c9fc7770752 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.385159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.402130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.402763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.403077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.403355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.404526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.406199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.407789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 96/193] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.408547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.440783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43241873-43f9-45b2-a37d-b42d03ecdcbb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.474933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.475780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.475780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.476053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.476777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=2afeb1cf31eb47f9a0c823030df9f4cb) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:05.477077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.478592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.484538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 98/194] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/role_assignments?user.id=2afeb1cf31eb47f9a0c823030df9f4cb&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.485148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.522784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.530065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24240a8b-dec2-4792-90e9-e3973f71ede6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.568157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.568823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.569083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.569475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] PATH_INFO: `/v3/users/fb4edf375c1c4f9baab91c6eecfe75af` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.585604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] RBAC: Authorizing `identity:get_user(user_id=fb4edf375c1c4f9baab91c6eecfe75af)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.587355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.593953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 97/195] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/users/fb4edf375c1c4f9baab91c6eecfe75af => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.593953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.610005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370074e5-1ffe-484e-8252-7266f15f8058 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.646323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.650937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.651584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.651865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.652860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] PATH_INFO: `/v3/users/b47769234c3d4e1a83613623aa9346f6` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.653347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] RBAC: Authorizing `identity:get_user(user_id=b47769234c3d4e1a83613623aa9346f6)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.654878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.660142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 99/196] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/users/b47769234c3d4e1a83613623aa9346f6 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.660142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.711538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4d92d-45e5-499d-b78e-9d7ee306bfe9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.746000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.746657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.746939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.747243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.747919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:05.748229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.749636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.758435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 98/197] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:05.759255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.771038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.774789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd843db-b500-4c79-9266-c37b8c5cf135 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.810404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.811047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.811328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.811603 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.812289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.813747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:05.865551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19537f-4cf0-4f31-9538-fefcdfc956d9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:05.872819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:05.900375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:05.900989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:05.901269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:05.901546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:05.924532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 100/198] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:05 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:05.934011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:05.980147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:05.981700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.062206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7f1e-db96-4ce2-9119-b4bf28df590b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.094802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 99/199] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:06.095759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:06.101051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.101665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.101994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.102263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.105209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:06.141107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:06.200604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.200604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.213896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a44d-a145-44e1-9a1a-8b9596e5fc61 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.261336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.262940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.263557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.263979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] PATH_INFO: `/v3/users/fb4edf375c1c4f9baab91c6eecfe75af` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.265853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] RBAC: Authorizing `identity:get_user(user_id=fb4edf375c1c4f9baab91c6eecfe75af)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.268230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.270798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 100/200] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/users/fb4edf375c1c4f9baab91c6eecfe75af => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:06.271600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:06.331877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 101/201] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:06.331877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:06.372009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:06.391516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:06.400818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158de3d-9bdb-4251-8d48-c246caf36c50 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.438242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fb5f9-a572-426a-bc79-3542ce9e8e4c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.444884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.445861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.446321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.446873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.448246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.450089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.452405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 101/202] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:06.453485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:06.480727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.481437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.481719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.482084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.560126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.561763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.572118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc31de4-1350-4c94-926a-daa828a8bdd9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.610993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.612105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.612590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.613374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.615014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.617128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.621316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 102/203] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:06.622479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:06.637930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:06.643056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:06.675681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 102/204] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:06.676881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:06.758520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a366ce1-f810-4c03-bec5-59c0ada071cc None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.791522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff596c6-ed08-4e30-80a4-594047980280 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.802812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.803630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.804365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.804941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.805984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=fb4edf375c1c4f9baab91c6eecfe75af) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:06.806438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.808568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.816370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 103/205] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/role_assignments?user.id=fb4edf375c1c4f9baab91c6eecfe75af&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:06.817518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:06.854052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.854838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.855221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.855594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.856440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] RBAC: Adding query filter params (role.id=89ffb3825da1475c8ae689ccea2455ea, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457, user.id=b47769234c3d4e1a83613623aa9346f6) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:06.856900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.858623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.872070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 103/206] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/role_assignments?user.id=b47769234c3d4e1a83613623aa9346f6&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=89ffb3825da1475c8ae689ccea2455ea => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:06.908999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:06.938934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c515a9-3541-42b2-9009-0ff763955dfb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:06.979422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800d707-1fb8-441a-b8e7-5b05e2add385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:06.980334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800d707-1fb8-441a-b8e7-5b05e2add385 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:06.980792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800d707-1fb8-441a-b8e7-5b05e2add385 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:06.981215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800d707-1fb8-441a-b8e7-5b05e2add385 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:06.982676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800d707-1fb8-441a-b8e7-5b05e2add385 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:06.984333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800d707-1fb8-441a-b8e7-5b05e2add385 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:06.986726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 104/207] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:06.993287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:07.028489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:07.141415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45443611-c23e-40d5-b4d7-4af5df3f43dd None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:07.177139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd4390-a5c3-4b14-8f88-e5b40c8b3624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:07.177751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd4390-a5c3-4b14-8f88-e5b40c8b3624 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:07.178053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd4390-a5c3-4b14-8f88-e5b40c8b3624 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:07.178325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd4390-a5c3-4b14-8f88-e5b40c8b3624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:07.179054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd4390-a5c3-4b14-8f88-e5b40c8b3624 None admin] RBAC: Adding query filter params (role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=fb4edf375c1c4f9baab91c6eecfe75af) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:07.179412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd4390-a5c3-4b14-8f88-e5b40c8b3624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:07.180947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd4390-a5c3-4b14-8f88-e5b40c8b3624 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:07.187129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 104/208] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:06 2022] GET /identity/v3/role_assignments?user.id=fb4edf375c1c4f9baab91c6eecfe75af&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:08.110793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a3c1a-c2d7-4e57-b387-78bab8079129 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.111812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a3c1a-c2d7-4e57-b387-78bab8079129 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.112840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a3c1a-c2d7-4e57-b387-78bab8079129 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.114707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 105/209] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:08.121029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef011f7-52c3-4be3-af2d-91c935f34376 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.122055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef011f7-52c3-4be3-af2d-91c935f34376 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.122557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef011f7-52c3-4be3-af2d-91c935f34376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.135992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8e57-c1c4-44b1-8698-e938bc028bdb None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.136305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8e57-c1c4-44b1-8698-e938bc028bdb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.136534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8e57-c1c4-44b1-8698-e938bc028bdb None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.138295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 106/210] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:08.145675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-bef011f7-52c3-4be3-af2d-91c935f34376 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:08.147534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bef011f7-52c3-4be3-af2d-91c935f34376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:08.148290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a16bbe-7644-4569-8fae-5679ffa578c6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.148576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a16bbe-7644-4569-8fae-5679ffa578c6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.148853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a16bbe-7644-4569-8fae-5679ffa578c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.171869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-18a16bbe-7644-4569-8fae-5679ffa578c6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:08.173167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-18a16bbe-7644-4569-8fae-5679ffa578c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:08.220254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 105/211] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:08.254357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 107/212] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:08.493146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef011f7-52c3-4be3-af2d-91c935f34376 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:08.508002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:08.542347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a16bbe-7644-4569-8fae-5679ffa578c6 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:08.622633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef011f7-52c3-4be3-af2d-91c935f34376 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:08.624119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:08.656835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a16bbe-7644-4569-8fae-5679ffa578c6 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:08.667512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:08.668462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.668995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.669492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.681052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:08.686652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:08.687528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:08.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:08.696229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 106/213] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:08.697568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:08.698830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:08.699481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.699759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.700030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.701155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:08.702700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:08.704365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 108/214] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:08.710582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:08.778559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:08.812208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86036ef-e4d8-47ad-b3c8-b60989b855dc None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:08.816722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0349fd9-f2d3-4a79-9019-0d5d02115354 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:08.851689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:08.852779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.853266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.853743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.854944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:08.855466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:08.855916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200430e-a109-4b41-bb07-74d067f9ef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:08.856536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200430e-a109-4b41-bb07-74d067f9ef10 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.856814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200430e-a109-4b41-bb07-74d067f9ef10 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.857083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200430e-a109-4b41-bb07-74d067f9ef10 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.857821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200430e-a109-4b41-bb07-74d067f9ef10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:08.858176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:08.859307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200430e-a109-4b41-bb07-74d067f9ef10 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:08.873126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:08.874000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 107/215] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:08.874740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:08.970047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 109/216] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:08 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:08.988047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82147-2e16-4593-94be-c74a77361619 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.989140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82147-2e16-4593-94be-c74a77361619 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.989427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82147-2e16-4593-94be-c74a77361619 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.991718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 110/217] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:08.998426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d3322-b676-4922-b5ff-cd0c31653c9c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:08.998729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d3322-b676-4922-b5ff-cd0c31653c9c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:08.998982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d3322-b676-4922-b5ff-cd0c31653c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:08.999189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f93e5-46cb-4fa9-aa03-14e6b4a2501d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:09.020824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-886d3322-b676-4922-b5ff-cd0c31653c9c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:09.022124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-886d3322-b676-4922-b5ff-cd0c31653c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:09.041972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:09.042746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.043032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.043703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] PATH_INFO: `/v3/users/a966c85e24e74272a7c2cd3f032b4153` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.046011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:09.064960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] RBAC: Authorizing `identity:get_user(user_id=a966c85e24e74272a7c2cd3f032b4153)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:09.067263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:09.069229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 108/218] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:08 2022] GET /identity/v3/users/a966c85e24e74272a7c2cd3f032b4153 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:09.077379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:09.113811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 111/219] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:09.159783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da61a-9061-4334-b29a-8a9b2abfd1da None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.160067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da61a-9061-4334-b29a-8a9b2abfd1da None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.160351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da61a-9061-4334-b29a-8a9b2abfd1da None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.161810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 112/220] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:09.167034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673670c-9df5-41e8-ae86-5a9eea80daaf None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.167351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673670c-9df5-41e8-ae86-5a9eea80daaf None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.167628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673670c-9df5-41e8-ae86-5a9eea80daaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.188103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2673670c-9df5-41e8-ae86-5a9eea80daaf None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:09.189304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2673670c-9df5-41e8-ae86-5a9eea80daaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:09.209931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c4c7d-2be2-4c1c-ae39-8aee20e952aa None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:09.253534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:09.254548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.254992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.255387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 113/221] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:09.339137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:09.344192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:09.345688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:09.396904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673670c-9df5-41e8-ae86-5a9eea80daaf None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:09.473538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 109/222] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:09.482826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:09.491483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:09.568044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673670c-9df5-41e8-ae86-5a9eea80daaf None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:09.634110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:09.634757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.635038 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.635312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.651939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:09.655316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:09.660814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:09.666708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 114/223] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:09.667654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:09.671121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ad9c4-1356-4c6e-834e-f397058502f4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:09.712781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:09.713611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.714040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.714403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.715747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:09.717264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:09.719377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 110/224] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:09 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:09.720295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:09.801316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255f273-6112-4da8-837c-827ebfb321bb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:09.819986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:09.841185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:09.842226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.842580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.842913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.844698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26f039-c290-480d-8d4f-1d909aa6c374 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:09.857441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:09.864679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:09.865682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:09.866556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:09.873794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 115/225] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:09.873794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:09.891613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:09.892303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:09.892525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:09.892804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:09.894012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:09.894324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:09.895780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:09.905421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 111/226] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:09.906319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:09.934142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:09.988566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff6e19-2221-4704-b3a7-a60e202c4280 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.016324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564d8c7-da2a-444e-a5db-f8364904e2f5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.027282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.028154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.028623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.029080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.030057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=a966c85e24e74272a7c2cd3f032b4153) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:10.030613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.032341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.038963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 116/227] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:09 2022] GET /identity/v3/role_assignments?user.id=a966c85e24e74272a7c2cd3f032b4153&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:10.040059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:10.078879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.082949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.083496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.085668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.086702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:10.087222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.089647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.111475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 112/228] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:10.124896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:10.142659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:10.238468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ded40-f099-4d84-ad45-17390b98f265 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.257918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:10.275638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733a780-08ea-4c65-85f4-fce259cb802e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.277476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.278370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.278869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.279335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] PATH_INFO: `/v3/users/fb4edf375c1c4f9baab91c6eecfe75af` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.280740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] RBAC: Authorizing `identity:get_user(user_id=fb4edf375c1c4f9baab91c6eecfe75af)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.282412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.284232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 117/229] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:10 2022] GET /identity/v3/users/fb4edf375c1c4f9baab91c6eecfe75af => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:10.289870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:10.366831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.367810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.368093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.368366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] PATH_INFO: `/v3/users/fb4edf375c1c4f9baab91c6eecfe75af` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.371581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] RBAC: Authorizing `identity:get_user(user_id=fb4edf375c1c4f9baab91c6eecfe75af)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.373054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.375101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 113/230] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:10 2022] GET /identity/v3/users/fb4edf375c1c4f9baab91c6eecfe75af => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:10.381211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:10.427551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e47c7-6d61-43d6-915d-da764f01b68d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.450100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:10.500137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.500823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.501106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.501561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.504437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.507552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.513248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 118/231] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:10 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:10.519516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:10.569890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c11f09-0ef4-41b4-8dd1-9e141ffcea9e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.632493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.633055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.633332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.633604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.634766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.636214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.637943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 114/232] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:10 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:10.644345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:10.669203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8284-9984-49ab-bca4-ed63cc011b53 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.686301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:10.712769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040191b-cd55-4a2d-a1cf-d5fa364aa0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.713794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040191b-cd55-4a2d-a1cf-d5fa364aa0d0 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.714207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040191b-cd55-4a2d-a1cf-d5fa364aa0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.714484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040191b-cd55-4a2d-a1cf-d5fa364aa0d0 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/fb4edf375c1c4f9baab91c6eecfe75af/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.716648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040191b-cd55-4a2d-a1cf-d5fa364aa0d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d0b552698c54af99ff7c548b8d1caeb, user_id=fb4edf375c1c4f9baab91c6eecfe75af, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.718393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040191b-cd55-4a2d-a1cf-d5fa364aa0d0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.797168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a121a76-c2f1-47f6-bb5b-c4580179bf6f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:10.797635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 119/233] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:10 2022] PUT /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/fb4edf375c1c4f9baab91c6eecfe75af/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:41:10.840282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483b331-5f6d-40f0-a2b3-3dbc807463ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:10.840977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483b331-5f6d-40f0-a2b3-3dbc807463ab None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:10.841257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483b331-5f6d-40f0-a2b3-3dbc807463ab None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:10.841531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483b331-5f6d-40f0-a2b3-3dbc807463ab None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/fb4edf375c1c4f9baab91c6eecfe75af/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:10.843897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483b331-5f6d-40f0-a2b3-3dbc807463ab None admin] RBAC: Authorizing `identity:create_grant(project_id=7d0b552698c54af99ff7c548b8d1caeb, user_id=fb4edf375c1c4f9baab91c6eecfe75af, role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:10.845414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483b331-5f6d-40f0-a2b3-3dbc807463ab None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:10.925472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 115/234] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:10 2022] PUT /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/fb4edf375c1c4f9baab91c6eecfe75af/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:41:11.351047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7c6a7-ee1e-4207-9d55-b95a4b8e5a6e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:11.352016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7c6a7-ee1e-4207-9d55-b95a4b8e5a6e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:11.352524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7c6a7-ee1e-4207-9d55-b95a4b8e5a6e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:11.354139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 120/235] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:11.360060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef8469-3241-4db5-bcb5-f76f60737609 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:11.360996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef8469-3241-4db5-bcb5-f76f60737609 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:11.361496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef8469-3241-4db5-bcb5-f76f60737609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:11.383210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-97ef8469-3241-4db5-bcb5-f76f60737609 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:11.384764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-97ef8469-3241-4db5-bcb5-f76f60737609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:11.474146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 116/236] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:11.498491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6c5e2-6fc1-4ccc-a1af-972b6a83db24 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:11.499094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6c5e2-6fc1-4ccc-a1af-972b6a83db24 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:11.499569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6c5e2-6fc1-4ccc-a1af-972b6a83db24 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:11.501179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 121/237] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:11.507406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fad75-5c9b-46ae-85ac-8b4219dc28cd None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:11.507964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fad75-5c9b-46ae-85ac-8b4219dc28cd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:11.508442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fad75-5c9b-46ae-85ac-8b4219dc28cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:11.531161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-712fad75-5c9b-46ae-85ac-8b4219dc28cd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:11.532680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-712fad75-5c9b-46ae-85ac-8b4219dc28cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:11.537181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:11.623956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 117/238] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:11.766264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6c5e2-6fc1-4ccc-a1af-972b6a83db24 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:11.811112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:11.894257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6c5e2-6fc1-4ccc-a1af-972b6a83db24 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:11.937604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:11.938276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:11.938552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:11.938821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:11.952397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:11.957568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:11.958536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:11.959489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 122/239] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:11.965726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fad75-5c9b-46ae-85ac-8b4219dc28cd None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:11.969710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:12.141661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fad75-5c9b-46ae-85ac-8b4219dc28cd None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:12.158231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:12.167610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13e91e-1ce7-4871-aa43-f4569856fdac None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:12.212867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:12.213886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:12.214381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:12.215354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:12.217030 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:12.219360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:12.223618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 118/240] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:12.228488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:12.247036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:12.248234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:12.248861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:12.249517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:12.250775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:12.251728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:12.253118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:12.256401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:12.281369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 123/241] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:12.284621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:12.432688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1447523-fe4c-44e6-949b-171f13ee4249 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:12.447618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eda318-a47d-4931-a6ea-6366f59dfbb9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:12.476248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61355ee2-0e8f-44dd-9e05-aff5a319b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:12.477237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61355ee2-0e8f-44dd-9e05-aff5a319b808 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:12.477692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61355ee2-0e8f-44dd-9e05-aff5a319b808 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:12.478160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61355ee2-0e8f-44dd-9e05-aff5a319b808 None admin] PATH_INFO: `/v3/users/c4fc030f9372495fbb1d2611fb603f1d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:12.496318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61355ee2-0e8f-44dd-9e05-aff5a319b808 None admin] RBAC: Authorizing `identity:get_user(user_id=c4fc030f9372495fbb1d2611fb603f1d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:12.498129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61355ee2-0e8f-44dd-9e05-aff5a319b808 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:12.500293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc3755-9415-43a4-b842-5b95043a99bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:12.500496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 124/242] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:12 2022] GET /identity/v3/users/c4fc030f9372495fbb1d2611fb603f1d => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:12.501096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc3755-9415-43a4-b842-5b95043a99bc None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:12.501442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc3755-9415-43a4-b842-5b95043a99bc None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:12.501851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc3755-9415-43a4-b842-5b95043a99bc None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:12.502341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7abd-dd5a-4054-af02-60d93203b48f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:12.503227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc3755-9415-43a4-b842-5b95043a99bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:12.503438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7abd-dd5a-4054-af02-60d93203b48f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:12.503842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7abd-dd5a-4054-af02-60d93203b48f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:12.505044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc3755-9415-43a4-b842-5b95043a99bc None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:12.505793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 125/243] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:12.506646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7abd-dd5a-4054-af02-60d93203b48f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:12.527922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:12.590045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 119/244] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:12 2022] POST /identity/v3/users => generated 293 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:41:12.592423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19315905-99a7-46fa-88e0-91acdd64e4e7 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:12.592824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19315905-99a7-46fa-88e0-91acdd64e4e7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:12.593225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19315905-99a7-46fa-88e0-91acdd64e4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:12.622018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-19315905-99a7-46fa-88e0-91acdd64e4e7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:12.622903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7abd-dd5a-4054-af02-60d93203b48f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:12.623732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-19315905-99a7-46fa-88e0-91acdd64e4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:12.663219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:12.664168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:12.664662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:12.665106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:12.713452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:12.725617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 120/245] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:12.761562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:12.763473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:12.783759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:12.894730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 126/246] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:12.900178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19315905-99a7-46fa-88e0-91acdd64e4e7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:13.020682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19315905-99a7-46fa-88e0-91acdd64e4e7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:13.059623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:13.060439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.060712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.060980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.062145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:13.063588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:13.068210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 121/247] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:12 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:13.071733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:13.093091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:13.177534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:13.197888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9093c-71a2-4808-aaab-9ef0a1216073 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:13.226137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:13.229020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b02d08-a395-4dbb-baab-eb67663588f2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:13.250868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06776ee5-7874-41c8-8100-04addbcfbe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:13.251852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776ee5-7874-41c8-8100-04addbcfbe80 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.252322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776ee5-7874-41c8-8100-04addbcfbe80 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.253337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776ee5-7874-41c8-8100-04addbcfbe80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.254371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06776ee5-7874-41c8-8100-04addbcfbe80 None admin] RBAC: Adding query filter params (role.id=89ffb3825da1475c8ae689ccea2455ea, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=c4fc030f9372495fbb1d2611fb603f1d) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:13.254869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06776ee5-7874-41c8-8100-04addbcfbe80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:13.256580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06776ee5-7874-41c8-8100-04addbcfbe80 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:13.264259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 127/248] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:13 2022] GET /identity/v3/role_assignments?user.id=c4fc030f9372495fbb1d2611fb603f1d&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=89ffb3825da1475c8ae689ccea2455ea => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:13.265288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22c25b-610e-4268-92d5-16cdb65a9c09 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.265574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22c25b-610e-4268-92d5-16cdb65a9c09 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.265871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22c25b-610e-4268-92d5-16cdb65a9c09 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.267169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 128/249] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:13.274038 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca86174-aa86-4591-b720-b0313c5fb186 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.274307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca86174-aa86-4591-b720-b0313c5fb186 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.274503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca86174-aa86-4591-b720-b0313c5fb186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.286510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:13.287174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.287459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.287729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.298716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-dca86174-aa86-4591-b720-b0313c5fb186 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:13.300039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dca86174-aa86-4591-b720-b0313c5fb186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:13.302735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:13.310196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:13.311159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:13.312378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 122/250] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:13.318828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:13.433821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 129/251] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:13.469168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e140685-c767-41db-b7fd-e948582ef386 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:13.496294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:13.508831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:13.509713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.510177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.510616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.511431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:13.511765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:13.513115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:13.520586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999310e-c085-4483-8dad-c071b47049f4 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.520869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999310e-c085-4483-8dad-c071b47049f4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.521138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999310e-c085-4483-8dad-c071b47049f4 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.524214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 130/252] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:13.526847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 123/253] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:13.529668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484b23-69f0-4744-b859-8b61059d309e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.529974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484b23-69f0-4744-b859-8b61059d309e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.530364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484b23-69f0-4744-b859-8b61059d309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.532795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:13.573292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-03484b23-69f0-4744-b859-8b61059d309e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:13.574537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-03484b23-69f0-4744-b859-8b61059d309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:13.686714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 131/254] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:13.712556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8458b34-8fde-44e2-9061-d5cd426db5fd None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:13.749327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:13.750003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.750279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.750549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] PATH_INFO: `/v3/users/a966c85e24e74272a7c2cd3f032b4153` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.751746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=a966c85e24e74272a7c2cd3f032b4153)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:13.753823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:13.755712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 124/255] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:13 2022] GET /identity/v3/users/a966c85e24e74272a7c2cd3f032b4153 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:13.762922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484b23-69f0-4744-b859-8b61059d309e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:13.824227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:13.900287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484b23-69f0-4744-b859-8b61059d309e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:13.936898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:13.937536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:13.937845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:13.938125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:13.939270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:13.940741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:13.942864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 132/256] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:13 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:13.947619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:13.950473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:14.033348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:14.104343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91004-91ca-4084-a32a-a0ee983ac3f0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:14.115256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30d6bf-3568-4d3c-86dc-b2c970da4c5e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:14.133897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:14.140123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:14.141023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:14.141495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:14.142012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:14.164733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:14.171436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:14.172320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:14.173109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:14.178709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 125/257] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:14.181153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:14.188021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:14.189001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:14.189482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:14.189985 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/a966c85e24e74272a7c2cd3f032b4153/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:14.194172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] RBAC: Authorizing `identity:create_grant(project_id=7d0b552698c54af99ff7c548b8d1caeb, user_id=a966c85e24e74272a7c2cd3f032b4153, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:14.200698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:14.273543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 133/258] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:13 2022] PUT /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/a966c85e24e74272a7c2cd3f032b4153/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:41:14.274918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:14.367947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b3ee-aac8-4ac2-956b-d6c9b37c00da None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:14.406155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:14.436710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93374a-8e55-4370-b982-41ae467c223e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:14.470559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:14.471231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:14.472063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:14.472341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:14.500270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:14.500907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:14.501187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:14.501460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:14.502164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:14.502464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:14.503883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:14.509518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:14.515024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:14.515876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:14.516755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 126/259] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:14.518674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 134/260] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:14.522626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:14.524003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:14.571169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:14.704063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f258-90d7-41b3-b4af-f42090b0cfcb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:14.704236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a181e1-7560-4844-9db1-d918467d9ff6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:14.772017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:14.772671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:14.772951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:14.773224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] PATH_INFO: `/v3/users/fb4edf375c1c4f9baab91c6eecfe75af` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:14.778786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] RBAC: Authorizing `identity:get_user(user_id=fb4edf375c1c4f9baab91c6eecfe75af)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:14.780301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:14.782268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 135/261] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:14 2022] GET /identity/v3/users/fb4edf375c1c4f9baab91c6eecfe75af => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:14.785258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:14.785915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:14.786192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:14.786464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:14.787465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:14.787680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:14.787789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:14.789270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:14.814670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 127/262] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:14.822909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:14.942137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:14.961957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05a9b7-acc5-47db-b7fa-6fff4ddf7e8e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:14.965757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:15.040185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:15.040808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.041085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.041353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.121302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eda906-5803-41a8-92a1-d22eac6b2cd2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:15.207435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:15.208088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.208359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.208461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:15.208616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] PATH_INFO: `/v3/users/fb4edf375c1c4f9baab91c6eecfe75af` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.209761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] RBAC: Authorizing `identity:get_user(user_id=fb4edf375c1c4f9baab91c6eecfe75af)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:15.209897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:15.211217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:15.213222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 128/263] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:14 2022] GET /identity/v3/users/fb4edf375c1c4f9baab91c6eecfe75af => generated 283 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:15.219417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:15.398932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e637dc-1ab2-461c-94e2-6dad4c07289e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:15.400524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:15.440943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:15.447301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8206278-a136-444a-a306-de74713739b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:15.447919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206278-a136-444a-a306-de74713739b3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.448199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206278-a136-444a-a306-de74713739b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.448465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206278-a136-444a-a306-de74713739b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.449148 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 136/264] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:41:15.455005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:15.529515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8206278-a136-444a-a306-de74713739b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:15.531276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8206278-a136-444a-a306-de74713739b3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:15.648412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 129/265] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:15.649262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659514-34bb-4b3d-b7e9-5236d51f26ae None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:15.654103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a1de-c78b-4eb2-93c3-82b85cd64487 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.655491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a1de-c78b-4eb2-93c3-82b85cd64487 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.655974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a1de-c78b-4eb2-93c3-82b85cd64487 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.659422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 130/266] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:15.660286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18a1de-c78b-4eb2-93c3-82b85cd64487 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:15.692667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:15.694539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff33c83-da55-4ac9-9e81-8f0f11c32ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:15.695376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33c83-da55-4ac9-9e81-8f0f11c32ced None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.695829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33c83-da55-4ac9-9e81-8f0f11c32ced None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.696273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33c83-da55-4ac9-9e81-8f0f11c32ced None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.697604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff33c83-da55-4ac9-9e81-8f0f11c32ced None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:15.699247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff33c83-da55-4ac9-9e81-8f0f11c32ced None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:15.701062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 137/267] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:15 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:15.702415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929489c1-3648-4085-b3df-88c56517c9ae None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.702904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929489c1-3648-4085-b3df-88c56517c9ae None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.703390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929489c1-3648-4085-b3df-88c56517c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.728967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-929489c1-3648-4085-b3df-88c56517c9ae None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:15.731215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-929489c1-3648-4085-b3df-88c56517c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:15.800530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:15.809327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 138/268] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:15.812661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929489c1-3648-4085-b3df-88c56517c9ae None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:15.814613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18a1de-c78b-4eb2-93c3-82b85cd64487 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:15.857138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98805045-26a9-40ec-ab52-db31b49f4e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:15.858260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805045-26a9-40ec-ab52-db31b49f4e37 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.858688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805045-26a9-40ec-ab52-db31b49f4e37 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.859129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805045-26a9-40ec-ab52-db31b49f4e37 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.861016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98805045-26a9-40ec-ab52-db31b49f4e37 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:15.863195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98805045-26a9-40ec-ab52-db31b49f4e37 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:15.866059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 131/269] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:15 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:15.872324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408ae1c-c7d4-4cdd-825e-2d5826e97424 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:15.873027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408ae1c-c7d4-4cdd-825e-2d5826e97424 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:15.873458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408ae1c-c7d4-4cdd-825e-2d5826e97424 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:15.875391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 132/270] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:15.876431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408ae1c-c7d4-4cdd-825e-2d5826e97424 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:15.983292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929489c1-3648-4085-b3df-88c56517c9ae None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:16.009565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:16.023381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a225aa4-f170-4a8a-ba7e-21ffc2800600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:16.024338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225aa4-f170-4a8a-ba7e-21ffc2800600 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.024824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225aa4-f170-4a8a-ba7e-21ffc2800600 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.025298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225aa4-f170-4a8a-ba7e-21ffc2800600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.026366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225aa4-f170-4a8a-ba7e-21ffc2800600 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=fb4edf375c1c4f9baab91c6eecfe75af) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:16.026961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225aa4-f170-4a8a-ba7e-21ffc2800600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:16.028672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225aa4-f170-4a8a-ba7e-21ffc2800600 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:16.035916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 139/271] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:15 2022] GET /identity/v3/role_assignments?user.id=fb4edf375c1c4f9baab91c6eecfe75af&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:16.037674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a6630-4a17-4067-aca8-b0774f901877 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.042475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a6630-4a17-4067-aca8-b0774f901877 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.042475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a6630-4a17-4067-aca8-b0774f901877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.058763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e66a6630-4a17-4067-aca8-b0774f901877 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:16.060485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e66a6630-4a17-4067-aca8-b0774f901877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:16.086132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408ae1c-c7d4-4cdd-825e-2d5826e97424 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:16.126436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:16.138713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 140/272] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:16.145857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:16.146490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.146763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.147031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.147777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] RBAC: Adding query filter params (role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=fb4edf375c1c4f9baab91c6eecfe75af) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:16.148078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:16.149557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:16.156157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a6630-4a17-4067-aca8-b0774f901877 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:16.156804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 133/273] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:15 2022] GET /identity/v3/role_assignments?user.id=fb4edf375c1c4f9baab91c6eecfe75af&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:16.301458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a6630-4a17-4067-aca8-b0774f901877 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:16.354565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58003730-cbc8-43b8-90ef-6e5150dda26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:16.355233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58003730-cbc8-43b8-90ef-6e5150dda26a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.355513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58003730-cbc8-43b8-90ef-6e5150dda26a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.355780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58003730-cbc8-43b8-90ef-6e5150dda26a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.357069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58003730-cbc8-43b8-90ef-6e5150dda26a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:16.358572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58003730-cbc8-43b8-90ef-6e5150dda26a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:16.360193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 141/274] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:16.366111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:16.480974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684827b-6ace-4798-bb08-7a827448c8af None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:16.516657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:16.517297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.517631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.517943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.519227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:16.520683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:16.527046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3ee7a-5f9b-411e-9e1a-b8b16cdcaf0c None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.527610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3ee7a-5f9b-411e-9e1a-b8b16cdcaf0c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.527610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3ee7a-5f9b-411e-9e1a-b8b16cdcaf0c None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.528895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 142/275] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:16.534363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:16.536626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b905652-e84b-473b-ab40-c5c9495706a2 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.537182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b905652-e84b-473b-ab40-c5c9495706a2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.537726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b905652-e84b-473b-ab40-c5c9495706a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.563480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9b905652-e84b-473b-ab40-c5c9495706a2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:16.564630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9b905652-e84b-473b-ab40-c5c9495706a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:16.631320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:16.645973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 143/276] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:16.646843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b905652-e84b-473b-ab40-c5c9495706a2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:16.656003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 134/277] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:16 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:16.766520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b905652-e84b-473b-ab40-c5c9495706a2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:16.803729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:16.804422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.804688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.804954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.815687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:16.820972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:16.821897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:16.822690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 144/278] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:16.827857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:16.942511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca9aab-36bf-43ac-9744-988b31c8f3c2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:16.943624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:16.983916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:16.985373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:16.985944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:16.986409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:16.987381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:16.987874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:16.989448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:17.000811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 135/279] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:17.007447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:17.013076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:17.082520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54460b22-fb24-43a1-95e3-68bd1f8a55ce None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:17.083018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:17.128937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f67dc-a65c-4f4f-b6ae-55b7e223fbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:17.130185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f67dc-a65c-4f4f-b6ae-55b7e223fbe6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.130664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f67dc-a65c-4f4f-b6ae-55b7e223fbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.131374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f67dc-a65c-4f4f-b6ae-55b7e223fbe6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.146025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f67dc-a65c-4f4f-b6ae-55b7e223fbe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:17.146962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcc91f-99d4-4442-8cbf-07cc5eba515a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-377f67dc-a65c-4f4f-b6ae-55b7e223fbe6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:17.152107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:17.153022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:17.153896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 145/280] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:17.155089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bbc43-d34e-4442-abae-d3c8c3c51084 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.155089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bbc43-d34e-4442-abae-d3c8c3c51084 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.155089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bbc43-d34e-4442-abae-d3c8c3c51084 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.156908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 146/281] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:17.163352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5e104-2f78-4787-9435-42b1528a6a5e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.163626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5e104-2f78-4787-9435-42b1528a6a5e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.164047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5e104-2f78-4787-9435-42b1528a6a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.183124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c6e5e104-2f78-4787-9435-42b1528a6a5e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:17.184315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c6e5e104-2f78-4787-9435-42b1528a6a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:17.202989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:17.204569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.205154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.205667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] PATH_INFO: `/v3/users/57c032e1e07a4cada62fa6f5747d3b4c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.224494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] RBAC: Authorizing `identity:get_user(user_id=57c032e1e07a4cada62fa6f5747d3b4c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:17.226394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:17.228592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 136/282] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:17 2022] GET /identity/v3/users/57c032e1e07a4cada62fa6f5747d3b4c => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:17.229699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:17.265812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 147/283] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:17.266744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5e104-2f78-4787-9435-42b1528a6a5e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:17.320855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:17.346802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f5883-8961-414c-8e89-d043a5d06953 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:17.360198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:17.387292 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:17.388408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.388890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.389458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.390400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:17.390907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:17.392694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:17.402859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 137/284] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:17.409983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:17.435265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5e104-2f78-4787-9435-42b1528a6a5e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:17.476401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:17.477370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.477880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.478357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.479940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, user.id=57c032e1e07a4cada62fa6f5747d3b4c, scope.system=all) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:17.480489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:17.482131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:17.489133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 148/285] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Sep 1 13:41:17 2022] GET /identity/v3/role_assignments?user.id=57c032e1e07a4cada62fa6f5747d3b4c&scope.system=all&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:17.569957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6992eb-c71e-4fcd-8260-45425211a6a6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:17.602223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:17.613230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:17.613896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.614170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.614440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] PATH_INFO: `/v3/users/c4fc030f9372495fbb1d2611fb603f1d` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.618695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] RBAC: Authorizing `identity:get_user(user_id=c4fc030f9372495fbb1d2611fb603f1d)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:17.622612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:17.626647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 138/286] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:17 2022] GET /identity/v3/users/c4fc030f9372495fbb1d2611fb603f1d => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:17.633407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:17.701393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:17.765240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ed92a-6f5d-41ab-bdc3-130530cb0a95 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:17.802243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:17.803243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.803735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.804257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.805745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:17.807631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:17.809843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 139/287] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:17 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:17.816704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:17.827692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1703b7-197d-4112-a01d-e0471f8870b6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:17.831214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:17.902762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:17.903792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:17.904310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:17.905626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:17.929433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:17.934935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:17.935813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:17.936704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:17.939707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 149/288] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:17.945253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:18.039591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a133d3-6e5c-4379-bdb4-0c447eb656c1 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:18.057973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:18.088028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7aa6d-67f4-4444-8b8e-b4c06ecbb165 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:18.104168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:18.104822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:18.105103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:18.105378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/c4fc030f9372495fbb1d2611fb603f1d/roles/89ffb3825da1475c8ae689ccea2455ea` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:18.111365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d0b552698c54af99ff7c548b8d1caeb, user_id=c4fc030f9372495fbb1d2611fb603f1d, role_id=89ffb3825da1475c8ae689ccea2455ea)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:18.112872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:18.130377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:18.131477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:18.131982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:18.132454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:18.133506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:18.134058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:18.135742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:18.147246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 150/289] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:18.153289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:18.183138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:18.204487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 140/290] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:17 2022] PUT /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/users/c4fc030f9372495fbb1d2611fb603f1d/roles/89ffb3825da1475c8ae689ccea2455ea => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:41:18.301542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d97d5-45dc-4ef3-919b-987f6c8a34e6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:18.353571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:18.354249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:18.354530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:18.354803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] PATH_INFO: `/v3/users/a966c85e24e74272a7c2cd3f032b4153` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:18.356010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=a966c85e24e74272a7c2cd3f032b4153)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:18.357453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:18.359403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 151/291] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:18 2022] GET /identity/v3/users/a966c85e24e74272a7c2cd3f032b4153 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:18.367991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:18.478884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dcf55-ef0c-4ca6-be04-5c4a31dfc1e3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:18.516027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:18.516644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:18.516925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:18.517200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:18.555131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:18.591664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:18.593187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:18.699798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 141/292] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:18.709345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:18.808994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:18.820232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d05d25-923b-459a-a673-e0749cab7ad8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:18.861046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66024089-fd30-4830-a170-c1c7bb92c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:18.861727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024089-fd30-4830-a170-c1c7bb92c545 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:18.862034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024089-fd30-4830-a170-c1c7bb92c545 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:18.862310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024089-fd30-4830-a170-c1c7bb92c545 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:18.866469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66024089-fd30-4830-a170-c1c7bb92c545 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:18.868213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66024089-fd30-4830-a170-c1c7bb92c545 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:18.869898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 152/293] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:18 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:18.876123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:18.947708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:18.984843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea6385-a10b-4d1d-91cf-3c9839619dbf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:19.029096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791cd63-3809-4388-a8d6-e8e53253eae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:19.029717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791cd63-3809-4388-a8d6-e8e53253eae6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.030055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791cd63-3809-4388-a8d6-e8e53253eae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.030329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791cd63-3809-4388-a8d6-e8e53253eae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.031073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791cd63-3809-4388-a8d6-e8e53253eae6 None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=a966c85e24e74272a7c2cd3f032b4153) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:19.031422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791cd63-3809-4388-a8d6-e8e53253eae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:19.032882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791cd63-3809-4388-a8d6-e8e53253eae6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:19.039595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 142/294] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:18 2022] GET /identity/v3/role_assignments?user.id=a966c85e24e74272a7c2cd3f032b4153&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:19.190686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcebbf5-f7c0-4bee-887d-b1fef221ac04 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.191058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcebbf5-f7c0-4bee-887d-b1fef221ac04 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.191254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcebbf5-f7c0-4bee-887d-b1fef221ac04 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.192908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 153/295] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:19.198324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ad658-d15f-4631-b347-8da7c0a41413 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.198599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ad658-d15f-4631-b347-8da7c0a41413 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.198943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ad658-d15f-4631-b347-8da7c0a41413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.219503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-842ad658-d15f-4631-b347-8da7c0a41413 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:19.220785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-842ad658-d15f-4631-b347-8da7c0a41413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:19.295728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 143/296] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:19.368745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58e612-a5dd-4fcd-90c8-2fd44f0fdbcc None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.369056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58e612-a5dd-4fcd-90c8-2fd44f0fdbcc None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.369302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58e612-a5dd-4fcd-90c8-2fd44f0fdbcc None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.370930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 154/297] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:19.376339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b63b2-7337-4a90-9204-750f35b3c0a3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.376787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b63b2-7337-4a90-9204-750f35b3c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.377212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b63b2-7337-4a90-9204-750f35b3c0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.397549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6f4b63b2-7337-4a90-9204-750f35b3c0a3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:19.399060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6f4b63b2-7337-4a90-9204-750f35b3c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:19.466295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:19.477042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 144/298] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:19.591941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58e612-a5dd-4fcd-90c8-2fd44f0fdbcc None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:19.712506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58e612-a5dd-4fcd-90c8-2fd44f0fdbcc None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:19.747475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:19.748029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.748306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.748574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.749668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:19.751139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:19.753570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 155/299] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:19.758791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b63b2-7337-4a90-9204-750f35b3c0a3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:19.770409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:19.805809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:19.868515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b63b2-7337-4a90-9204-750f35b3c0a3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:19.879301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f77e7e-6b04-43db-9c5d-67a94824e2f9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:19.905614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11267d04-39d5-4c49-a5d9-5ee1cde605a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:19.906370 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11267d04-39d5-4c49-a5d9-5ee1cde605a9 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.906748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11267d04-39d5-4c49-a5d9-5ee1cde605a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.907029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11267d04-39d5-4c49-a5d9-5ee1cde605a9 None admin] PATH_INFO: `/v3/groups` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.907677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11267d04-39d5-4c49-a5d9-5ee1cde605a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:19.909780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11267d04-39d5-4c49-a5d9-5ee1cde605a9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:19.915153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:19.916230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:19.916513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:19.916983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:19.928434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:19.933462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:19.934404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:19.935259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:19.936397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 156/300] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:19.940250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.014657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 145/301] 10.222.0.61 () {60 vars in 1134 bytes} [Thu Sep 1 13:41:19 2022] POST /identity/v3/groups => generated 229 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:20.015930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478aeb4-f8ca-4511-b360-e99f687e5706 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.016363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478aeb4-f8ca-4511-b360-e99f687e5706 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.016853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478aeb4-f8ca-4511-b360-e99f687e5706 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.018057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 146/302] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:20.023979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f1e1e-7f90-4f0a-9333-f12833535a1d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.024267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f1e1e-7f90-4f0a-9333-f12833535a1d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.024632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f1e1e-7f90-4f0a-9333-f12833535a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.045100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-431f1e1e-7f90-4f0a-9333-f12833535a1d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:20.046398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-431f1e1e-7f90-4f0a-9333-f12833535a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:20.107060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:20.126212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:20.141802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 147/303] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:20.142158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb694f4b-27cd-4d3d-a362-889d2319961b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:20.186244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:20.186867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.187205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.187481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.188217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:20.188527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:20.190033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:20.199181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 157/304] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:20.204653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f1e1e-7f90-4f0a-9333-f12833535a1d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.322972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f1e1e-7f90-4f0a-9333-f12833535a1d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:20.359912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:20.360554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.360830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.361101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] PATH_INFO: `/v3/users/fbde2e7ad918437bbaee3ae18b123dc3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.375502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] RBAC: Authorizing `identity:get_user(user_id=fbde2e7ad918437bbaee3ae18b123dc3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:20.377037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:20.378761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 148/305] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:20 2022] GET /identity/v3/users/fbde2e7ad918437bbaee3ae18b123dc3 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:20.384821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.421448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.467270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:20.498133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242d1ec-a8fd-4432-a337-289561d24055 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:20.529109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3bb2b-f5c0-429f-bd41-05f03ed06c44 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:20.534537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25d380-30f0-45ac-aaf8-3e2ad815beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:20.535145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25d380-30f0-45ac-aaf8-3e2ad815beee None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.535422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25d380-30f0-45ac-aaf8-3e2ad815beee None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.535692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25d380-30f0-45ac-aaf8-3e2ad815beee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.536421 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25d380-30f0-45ac-aaf8-3e2ad815beee None admin] RBAC: Adding query filter params (role.id=89ffb3825da1475c8ae689ccea2455ea, user.id=fbde2e7ad918437bbaee3ae18b123dc3, scope.system=all) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:20.536725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25d380-30f0-45ac-aaf8-3e2ad815beee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:20.538180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25d380-30f0-45ac-aaf8-3e2ad815beee None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:20.543348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 158/306] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Sep 1 13:41:20 2022] GET /identity/v3/role_assignments?user.id=fbde2e7ad918437bbaee3ae18b123dc3&scope.system=all&role.id=89ffb3825da1475c8ae689ccea2455ea => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:20.543348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e2c86-a155-42ff-849f-10070c7d61dc None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.543348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e2c86-a155-42ff-849f-10070c7d61dc None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.543348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e2c86-a155-42ff-849f-10070c7d61dc None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.546722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 159/307] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:20.552792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe08ce-4923-43c3-b6eb-ac311b20fc5b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.553079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe08ce-4923-43c3-b6eb-ac311b20fc5b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.553354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe08ce-4923-43c3-b6eb-ac311b20fc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.574813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-dafe08ce-4923-43c3-b6eb-ac311b20fc5b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:20.575958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dafe08ce-4923-43c3-b6eb-ac311b20fc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:20.586509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:20.587157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.587431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.587697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.591512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:20.610119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:20.618438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:20.619391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:20.620175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:20.625846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 149/308] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:20.627807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.673553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 160/309] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:20.751060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be64a2-dda1-4d04-9307-8daa83eae2ec None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:20.754072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:20.787338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:20.788078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.788257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.788530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.789200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:20.789499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:20.790953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:20.799772 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 150/310] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:20.805576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe08ce-4923-43c3-b6eb-ac311b20fc5b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.911338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe08ce-4923-43c3-b6eb-ac311b20fc5b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:20.948793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:20.949433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:20.949705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:20.950004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] PATH_INFO: `/v3/users/57c032e1e07a4cada62fa6f5747d3b4c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:20.951137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] RBAC: Authorizing `identity:get_user(user_id=57c032e1e07a4cada62fa6f5747d3b4c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:20.952568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:20.954518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 161/311] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:20 2022] GET /identity/v3/users/57c032e1e07a4cada62fa6f5747d3b4c => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:20.960680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.961084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:20.979686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:21.090003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461b8c-bb4c-4f5b-895f-016c9a1976b8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:21.090782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb692ff-b6bb-4f0c-96f0-41f0eaf21dd4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:21.134652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:21.134652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:21.134652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:21.134652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:21.136253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:21.137229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:21.137516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:21.137810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] PATH_INFO: `/v3/system/users/57c032e1e07a4cada62fa6f5747d3b4c/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:21.139366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=57c032e1e07a4cada62fa6f5747d3b4c, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:21.141351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:21.146078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:21.153048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:21.161521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 151/312] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:21.162380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:21.191630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:21.259267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 162/313] 10.222.0.61 () {58 vars in 1334 bytes} [Thu Sep 1 13:41:20 2022] PUT /identity/v3/system/users/57c032e1e07a4cada62fa6f5747d3b4c/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:21.281505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4faf9-bbec-4658-ba9b-7e606a207345 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:21.323926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:21.324551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:21.324826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:21.325092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:21.325867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:21.326171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:21.327753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:21.339278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 152/314] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:21.344638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:21.436083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:21.458827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d322e5d-9be4-4794-97ad-bd1c6af1a348 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:21.495036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:21.495687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:21.495963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:21.496237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] PATH_INFO: `/v3/users/c4fc030f9372495fbb1d2611fb603f1d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:21.497396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] RBAC: Authorizing `identity:get_user(user_id=c4fc030f9372495fbb1d2611fb603f1d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:21.498863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:21.500842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 163/315] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:21 2022] GET /identity/v3/users/c4fc030f9372495fbb1d2611fb603f1d => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:21.506656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:21.597942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:21.614991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cca7d-60e8-4c30-b842-f0a6ea4b2484 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:21.653977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:21.654592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:21.654867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:21.655137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:21.727160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:21.728557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:21.814276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:21.831694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 153/316] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:21.837426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:21.947700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea32a94-e4dd-4df3-9b9b-9c09d06e6bbb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:21.983875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fc954-0333-40f3-a48b-450ebc4099fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:21.984921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fc954-0333-40f3-a48b-450ebc4099fd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:21.985505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fc954-0333-40f3-a48b-450ebc4099fd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:21.986109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fc954-0333-40f3-a48b-450ebc4099fd None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:21.987995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fc954-0333-40f3-a48b-450ebc4099fd None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:21.989604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fc954-0333-40f3-a48b-450ebc4099fd None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:21.991998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 164/317] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:21 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:21.997871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:22.079009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19dcf1-0d60-4ae1-8f49-8be51a6461cb None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:22.079325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19dcf1-0d60-4ae1-8f49-8be51a6461cb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:22.079563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19dcf1-0d60-4ae1-8f49-8be51a6461cb None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:22.082260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 165/318] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:22.087362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118b94a-4599-4168-a136-cd3850bc8905 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:22.087650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118b94a-4599-4168-a136-cd3850bc8905 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:22.087928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118b94a-4599-4168-a136-cd3850bc8905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:22.102672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1ee42-18ec-440d-bd3a-b041036cbe24 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:22.107142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a118b94a-4599-4168-a136-cd3850bc8905 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:22.108573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a118b94a-4599-4168-a136-cd3850bc8905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:22.118122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:22.140341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:22.140951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:22.141228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:22.141505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:22.142343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] RBAC: Adding query filter params (role.id=89ffb3825da1475c8ae689ccea2455ea, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb, user.id=c4fc030f9372495fbb1d2611fb603f1d) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:22.142649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:22.144105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:22.150522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 154/319] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:21 2022] GET /identity/v3/role_assignments?user.id=c4fc030f9372495fbb1d2611fb603f1d&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=89ffb3825da1475c8ae689ccea2455ea => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:22.192515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 166/320] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:22.477688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:22.490754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae78b4-171d-46ec-8e42-3e8179ff510a None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:22.491059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae78b4-171d-46ec-8e42-3e8179ff510a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:22.491322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae78b4-171d-46ec-8e42-3e8179ff510a None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:22.492928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 167/321] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:22.499429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796458b-33b1-4797-a6b0-7e567d915b05 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:22.499712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796458b-33b1-4797-a6b0-7e567d915b05 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:22.500019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796458b-33b1-4797-a6b0-7e567d915b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:22.518321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a796458b-33b1-4797-a6b0-7e567d915b05 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:22.519757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a796458b-33b1-4797-a6b0-7e567d915b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:22.529215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:22.582061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d8632-a56e-49f5-ae3b-ecc23d05ddef None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:22.592324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 168/322] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:22.616844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:22.617762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:22.618040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:22.618354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:22.619455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:22.620900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:22.622678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 155/323] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:22.628370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796458b-33b1-4797-a6b0-7e567d915b05 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:22.704228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:22.734610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796458b-33b1-4797-a6b0-7e567d915b05 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:22.769792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:22.770435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:22.770707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:22.770971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] PATH_INFO: `/v3/groups` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:22.771612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:22.773034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:22.863099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:22.886044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 169/324] 10.222.0.61 () {60 vars in 1134 bytes} [Thu Sep 1 13:41:22 2022] POST /identity/v3/groups => generated 226 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:22.986204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e3edd-89a6-4d2e-8faf-ad00427e885e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:23.005155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:23.025100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:23.025733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:23.026033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:23.026300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:23.037461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:23.042384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:23.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:23.044055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 156/325] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:23.049401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:23.165283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fa883-919e-4c64-a1fb-1b10d1f2f13d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:23.197717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:23.203820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb468b72-8304-4334-a222-72718b5064c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:23.204513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb468b72-8304-4334-a222-72718b5064c7 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:23.204829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb468b72-8304-4334-a222-72718b5064c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:23.205110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb468b72-8304-4334-a222-72718b5064c7 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:23.206722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb468b72-8304-4334-a222-72718b5064c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:23.207031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb468b72-8304-4334-a222-72718b5064c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:23.208458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb468b72-8304-4334-a222-72718b5064c7 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:23.218363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 170/326] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:23.223902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:23.307610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb56cf-5c49-4bc6-8115-e2e3c508b5ae None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:23.307936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb56cf-5c49-4bc6-8115-e2e3c508b5ae None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:23.308161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb56cf-5c49-4bc6-8115-e2e3c508b5ae None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:23.310996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 171/327] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:23.321490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e79d04-c78a-4a4c-9168-a6fd430e4467 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:23.321847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e79d04-c78a-4a4c-9168-a6fd430e4467 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:23.322090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e79d04-c78a-4a4c-9168-a6fd430e4467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:23.344251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-95e79d04-c78a-4a4c-9168-a6fd430e4467 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:23.345469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-95e79d04-c78a-4a4c-9168-a6fd430e4467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:23.347997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fab9a7-3bc1-4fea-ac6e-178a772e6e1c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:23.401102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:23.402048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:23.402450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:23.402842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] PATH_INFO: `/v3/users/fbde2e7ad918437bbaee3ae18b123dc3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:23.404335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] RBAC: Authorizing `identity:get_user(user_id=fbde2e7ad918437bbaee3ae18b123dc3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:23.406449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:23.408827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 157/328] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:23 2022] GET /identity/v3/users/fbde2e7ad918437bbaee3ae18b123dc3 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:23.414929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:23.448514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 172/329] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:23.514168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:23.593105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dd3bd-8a9f-40fb-a5fc-a4b2d473b72a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:23.630539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:23.631204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:23.631577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:23.631853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] PATH_INFO: `/v3/system/users/fbde2e7ad918437bbaee3ae18b123dc3/roles/89ffb3825da1475c8ae689ccea2455ea` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:23.633323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fbde2e7ad918437bbaee3ae18b123dc3, role_id=89ffb3825da1475c8ae689ccea2455ea)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:23.634779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:23.694826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 158/330] 10.222.0.61 () {58 vars in 1334 bytes} [Thu Sep 1 13:41:23 2022] PUT /identity/v3/system/users/fbde2e7ad918437bbaee3ae18b123dc3/roles/89ffb3825da1475c8ae689ccea2455ea => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:23.830890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e79d04-c78a-4a4c-9168-a6fd430e4467 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:23.972405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e79d04-c78a-4a4c-9168-a6fd430e4467 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:23.998921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:24.013269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:24.013948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:24.014231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:24.014505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:24.025982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:24.031296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:24.032189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:24.032971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 173/331] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:24.041110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:24.190609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32b4c4-4e34-409d-af46-3572f52b42a9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:24.201528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:24.231565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:24.232202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:24.238167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:24.238449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:24.239161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:24.239469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:24.240876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:24.251623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 159/332] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:24.257622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:24.400101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425765e-d6e0-482e-8280-7e5928a24bb9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:24.445191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028e475-3b90-4a0b-8aff-fdf020b8f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:24.445793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028e475-3b90-4a0b-8aff-fdf020b8f378 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:24.446310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028e475-3b90-4a0b-8aff-fdf020b8f378 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:24.446639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028e475-3b90-4a0b-8aff-fdf020b8f378 None admin] PATH_INFO: `/v3/users/57c032e1e07a4cada62fa6f5747d3b4c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:24.447974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028e475-3b90-4a0b-8aff-fdf020b8f378 None admin] RBAC: Authorizing `identity:get_user(user_id=57c032e1e07a4cada62fa6f5747d3b4c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:24.449433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028e475-3b90-4a0b-8aff-fdf020b8f378 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:24.451380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 174/333] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:24 2022] GET /identity/v3/users/57c032e1e07a4cada62fa6f5747d3b4c => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:24.458797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:24.594980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a58447-df45-4714-ad58-0aaa3bdfbc0c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:24.645868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d650cfd-1aff-4d2e-bd10-5eecc6208572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:24.646527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650cfd-1aff-4d2e-bd10-5eecc6208572 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:24.646876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650cfd-1aff-4d2e-bd10-5eecc6208572 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:24.647161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650cfd-1aff-4d2e-bd10-5eecc6208572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:24.647910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d650cfd-1aff-4d2e-bd10-5eecc6208572 None admin] RBAC: Adding query filter params (role.id=9f2bf5a138d54952a913a69eb59fa144, user.id=57c032e1e07a4cada62fa6f5747d3b4c, scope.system=all) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:24.648218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d650cfd-1aff-4d2e-bd10-5eecc6208572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:24.649675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d650cfd-1aff-4d2e-bd10-5eecc6208572 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:24.668886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 160/334] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Sep 1 13:41:24 2022] GET /identity/v3/role_assignments?user.id=57c032e1e07a4cada62fa6f5747d3b4c&scope.system=all&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:25.472711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed682-414a-440a-98a8-2f29eafcef6c None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:25.473206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed682-414a-440a-98a8-2f29eafcef6c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:25.473279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed682-414a-440a-98a8-2f29eafcef6c None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:25.474894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 175/335] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:25.487459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3f656-a17d-463a-933f-92c8ad4da4c9 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:25.487939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3f656-a17d-463a-933f-92c8ad4da4c9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:25.488013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3f656-a17d-463a-933f-92c8ad4da4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:25.517067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-fff3f656-a17d-463a-933f-92c8ad4da4c9 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:25.517067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fff3f656-a17d-463a-933f-92c8ad4da4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:25.543263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:25.597043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 161/336] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:25.935998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3ccc6-6561-42d4-9769-3816c139a145 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:25.936377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3ccc6-6561-42d4-9769-3816c139a145 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:25.936554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3ccc6-6561-42d4-9769-3816c139a145 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:25.938160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 176/337] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:25.944022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690c2d1-237c-451d-8d6c-8df4de717233 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:25.944342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690c2d1-237c-451d-8d6c-8df4de717233 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:25.944617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690c2d1-237c-451d-8d6c-8df4de717233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:25.970868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6690c2d1-237c-451d-8d6c-8df4de717233 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:25.972186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6690c2d1-237c-451d-8d6c-8df4de717233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:26.053108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 162/338] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:26.178593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3ccc6-6561-42d4-9769-3816c139a145 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:26.243910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:26.301758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3ccc6-6561-42d4-9769-3816c139a145 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:26.341801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e6691-5202-4a34-ba50-1881267634a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:26.342449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e6691-5202-4a34-ba50-1881267634a4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:26.342731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e6691-5202-4a34-ba50-1881267634a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:26.343006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e6691-5202-4a34-ba50-1881267634a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:26.355146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e6691-5202-4a34-ba50-1881267634a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-2b1e6691-5202-4a34-ba50-1881267634a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:26.360820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:26.362333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:26.363577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 177/339] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:26.369322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690c2d1-237c-451d-8d6c-8df4de717233 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:26.474912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:26.476955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dde54-d22f-4047-a81d-fe7ccaa145b8 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:26.477482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dde54-d22f-4047-a81d-fe7ccaa145b8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:26.477482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dde54-d22f-4047-a81d-fe7ccaa145b8 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:26.479050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 178/340] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:26.487589 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73ba23-19cd-4546-80c7-a8aa8fff962c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:26.488330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73ba23-19cd-4546-80c7-a8aa8fff962c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:26.488834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73ba23-19cd-4546-80c7-a8aa8fff962c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:26.493577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690c2d1-237c-451d-8d6c-8df4de717233 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:26.518630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2b73ba23-19cd-4546-80c7-a8aa8fff962c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:26.520864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2b73ba23-19cd-4546-80c7-a8aa8fff962c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:26.558781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:26.559921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:26.560416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:26.560896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:26.561836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:26.562407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:26.564186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:26.582317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 163/341] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:26.588713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:26.632415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 179/342] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:26.632415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73ba23-19cd-4546-80c7-a8aa8fff962c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:26.690115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:26.709054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfde14-b133-4eb6-800f-6de6ed8612b0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:26.748542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:26.749637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:26.750170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:26.750665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:26.770941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:26.784542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73ba23-19cd-4546-80c7-a8aa8fff962c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:26.827296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:26.829199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:26.876840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:26.877677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:26.877971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:26.878244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:26.910771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:26.915714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:26.916666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:26.918158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 180/343] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:26.922883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:26.953837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 164/344] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:26.959764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:27.069125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:27.170662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57361e-1fdf-44dc-9494-593088ead55a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:27.172705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:27.182920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aae5ac-eb3a-4e3a-b80a-f0b593cecf23 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:27.229810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:27.230471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.230753 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.231031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.231765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:27.232284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:27.236292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:27.240958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0118f-76ab-4af8-8ac9-19ad53a66d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:27.243405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0118f-76ab-4af8-8ac9-19ad53a66d2d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.243990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0118f-76ab-4af8-8ac9-19ad53a66d2d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.244471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0118f-76ab-4af8-8ac9-19ad53a66d2d None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.246113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0118f-76ab-4af8-8ac9-19ad53a66d2d None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:27.246519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 181/345] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:27.247470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:27.248474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0118f-76ab-4af8-8ac9-19ad53a66d2d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:27.255300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 165/346] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:26 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:27.257183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bed1-51bf-46c1-ac1b-ea2a107bc004 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.257764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bed1-51bf-46c1-ac1b-ea2a107bc004 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.258290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102bed1-51bf-46c1-ac1b-ea2a107bc004 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.261418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 166/347] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:27.262530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bed1-51bf-46c1-ac1b-ea2a107bc004 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:27.391402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae6d5e-73f0-47f0-a6d2-9eb98d352a43 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:27.414327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102bed1-51bf-46c1-ac1b-ea2a107bc004 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:27.432370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7ca4e-fbe2-4bc1-baaf-89dcdd42e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:27.433374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ca4e-fbe2-4bc1-baaf-89dcdd42e358 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.433911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ca4e-fbe2-4bc1-baaf-89dcdd42e358 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.434421 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ca4e-fbe2-4bc1-baaf-89dcdd42e358 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.437694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:27.449068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7ca4e-fbe2-4bc1-baaf-89dcdd42e358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-2fa7ca4e-fbe2-4bc1-baaf-89dcdd42e358 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:27.455975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:27.456919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:27.457749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:27.458907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:27.459487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.460179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.460179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.461913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 182/348] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:27.463473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369d4c-324e-4929-b4a7-2eac7ee951d1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.463971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369d4c-324e-4929-b4a7-2eac7ee951d1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.464460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369d4c-324e-4929-b4a7-2eac7ee951d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.488856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2d369d4c-324e-4929-b4a7-2eac7ee951d1 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:27.490950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2d369d4c-324e-4929-b4a7-2eac7ee951d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:27.512764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:27.565884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:27.568140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:27.572955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 183/349] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:27.574210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369d4c-324e-4929-b4a7-2eac7ee951d1 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:27.743889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369d4c-324e-4929-b4a7-2eac7ee951d1 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:27.756708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 167/350] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:27.762204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:27.785735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:27.804098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073390c-72a9-4735-acfb-6df224d93845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:27.805101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073390c-72a9-4735-acfb-6df224d93845 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.805552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073390c-72a9-4735-acfb-6df224d93845 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.806049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073390c-72a9-4735-acfb-6df224d93845 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.816491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073390c-72a9-4735-acfb-6df224d93845 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:27.818159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073390c-72a9-4735-acfb-6df224d93845 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:27.822821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 184/351] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:27 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:27.822821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaff97-edae-48ac-bc19-34f70207da82 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.822821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaff97-edae-48ac-bc19-34f70207da82 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.822821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaff97-edae-48ac-bc19-34f70207da82 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.823489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 185/352] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:27.824199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaff97-edae-48ac-bc19-34f70207da82 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:27.845284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:27.905355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86243bc-d358-43e0-8bb8-90363b83e073 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:27.947153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2848-eb34-409f-ae84-8e789d7e3067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:27.948215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2848-eb34-409f-ae84-8e789d7e3067 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.950999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2848-eb34-409f-ae84-8e789d7e3067 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.950999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2848-eb34-409f-ae84-8e789d7e3067 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.950999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2848-eb34-409f-ae84-8e789d7e3067 None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:27.950999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2848-eb34-409f-ae84-8e789d7e3067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:27.951290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2848-eb34-409f-ae84-8e789d7e3067 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:27.962262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 168/353] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:27.962507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaff97-edae-48ac-bc19-34f70207da82 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:27.967861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47146f14-05b5-427b-aee1-494ec382a513 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:27.968732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47146f14-05b5-427b-aee1-494ec382a513 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:27.968732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47146f14-05b5-427b-aee1-494ec382a513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:27.990865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-47146f14-05b5-427b-aee1-494ec382a513 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:27.992453 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-47146f14-05b5-427b-aee1-494ec382a513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:28.009048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.009627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.009969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.010286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.011630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:28.013231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:28.015280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 186/354] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:27 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:28.016202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:28.087902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 169/355] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:28.088797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47146f14-05b5-427b-aee1-494ec382a513 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:28.112300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:28.139308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:28.159030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795820-0f66-404c-ba3e-7f4a55568351 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:28.211057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.211679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.211973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.212225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.212965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:28.213270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:28.214727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:28.221292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 187/356] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:28.226103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:28.234070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47146f14-05b5-427b-aee1-494ec382a513 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:28.281826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.282467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.282750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.283020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.296519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:28.306990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:28.307971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:28.309240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 170/357] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:28.310165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:28.355127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b186c-13bb-415b-a3f3-b7f88204e25b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:28.407351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.408545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.408871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.409117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.437710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:28.438523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:28.483331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482ce27-0c03-48c5-bb50-fff9685c0b78 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:28.512692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:28.514192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:28.529473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fde160-038b-4dbc-85dd-582ca3e78dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.530799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fde160-038b-4dbc-85dd-582ca3e78dca None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.531346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fde160-038b-4dbc-85dd-582ca3e78dca None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.531689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fde160-038b-4dbc-85dd-582ca3e78dca None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.534236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fde160-038b-4dbc-85dd-582ca3e78dca None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:28.536942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fde160-038b-4dbc-85dd-582ca3e78dca None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:28.540726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 171/358] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:28.540726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec39da5-2b1f-4610-a964-516491169597 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.540726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec39da5-2b1f-4610-a964-516491169597 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.540726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec39da5-2b1f-4610-a964-516491169597 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.541222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 172/359] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:28.544146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec39da5-2b1f-4610-a964-516491169597 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:28.656524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 188/360] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:28.662958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:28.685610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec39da5-2b1f-4610-a964-516491169597 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:28.725723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:28.737171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.740129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.740129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.740129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.740129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:28.740129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:28.740959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:28.748584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:28.754118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 173/361] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:28.755026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:28.805919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42944b3-f0e0-4cd3-b8fb-7cadb8554c90 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:28.853302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ad6e6-f08c-41da-95fe-9a3a92e44ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.854686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ad6e6-f08c-41da-95fe-9a3a92e44ddd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.855190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ad6e6-f08c-41da-95fe-9a3a92e44ddd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.856244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ad6e6-f08c-41da-95fe-9a3a92e44ddd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.876925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ad6e6-f08c-41da-95fe-9a3a92e44ddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-1c0ad6e6-f08c-41da-95fe-9a3a92e44ddd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:28.884461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:28.886206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 189/362] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:28.886996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aae4ad-263f-490c-b4cf-d6df8b4018bc None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.887706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aae4ad-263f-490c-b4cf-d6df8b4018bc None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.887706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aae4ad-263f-490c-b4cf-d6df8b4018bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.907584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2d350-b52f-4f3e-92a3-61ecfd3608a5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:28.909415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f4aae4ad-263f-490c-b4cf-d6df8b4018bc None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:28.910675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f4aae4ad-263f-490c-b4cf-d6df8b4018bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:28.978142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:28.979008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:28.979430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:28.979851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:28.980601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:28.980909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:28.983626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:28.991181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 174/363] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:28.995754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.018609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 190/364] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:29.019774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aae4ad-263f-490c-b4cf-d6df8b4018bc None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.077797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:29.126159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:29.165304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aae4ad-263f-490c-b4cf-d6df8b4018bc None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:29.170937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3810a80-c297-4ded-9570-74e141b4c1db None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:29.220241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:29.221985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:29.223587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:29.224815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:29.230068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:29.231576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:29.236703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:29.237363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:29.237644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:29.237944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] PATH_INFO: `/v3/users/fbde2e7ad918437bbaee3ae18b123dc3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:29.239295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] RBAC: Authorizing `identity:get_user(user_id=fbde2e7ad918437bbaee3ae18b123dc3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:29.240729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:29.242601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 175/365] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:28 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:29.243517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.244251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 191/366] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/users/fbde2e7ad918437bbaee3ae18b123dc3 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:29.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.365839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40afd-5ac2-48a4-a1ce-bf6e736ec2e5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:29.396340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:29.401214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:29.401853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:29.402130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:29.402397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:29.403072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:29.403431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:29.404805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:29.417807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 176/367] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:29.417807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.444255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b82a6-0d80-4213-9b6b-bd5fdfdabd22 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:29.468488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:29.516343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:29.518422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:29.518921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:29.519405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:29.520792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:29.522572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:29.524973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 192/368] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:29.526057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.531012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165034c-78b4-4a7d-8155-ee5e694ddc20 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:29.566269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:29.566880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:29.567154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:29.567425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:29.568166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] RBAC: Adding query filter params (role.id=89ffb3825da1475c8ae689ccea2455ea, user.id=fbde2e7ad918437bbaee3ae18b123dc3, scope.system=all) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:29.568479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:29.571636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:29.590208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 177/369] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/role_assignments?user.id=fbde2e7ad918437bbaee3ae18b123dc3&scope.system=all&role.id=89ffb3825da1475c8ae689ccea2455ea => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:29.591190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.661550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e279e5-a0eb-4a6a-9b89-1f6379aa7d2e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:29.706382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:29.707377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:29.708443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:29.708934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:29.735854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:29.797938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:29.805196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f05052-27d5-4451-9ef9-9b4c42ca2def None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:29.841645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:29.844026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:29.852858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:29.853560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:29.853873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:29.854153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:29.870442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:29.877471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:29.879283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 178/370] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:29.882132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:29.983303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 193/371] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:29.984615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:30.027391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f5b67-340d-49ca-9788-1c2780be549f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:30.042581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:30.066962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:30.067753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.068031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.068308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.069178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:30.069522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:30.071139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:30.080119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 179/372] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:30.086757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:30.088333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:30.111897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5f31a-6276-453c-8fd0-d15bfe74ee1e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:30.192552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:30.193457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.194083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.194506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.195417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:30.195882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:30.197606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:30.212455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 194/373] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:30.218659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:30.272113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b3ceb-92fd-4205-9d10-78cc2aa1b02a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:30.327228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:30.327881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.328156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.328426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.334597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:30.336080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:30.337803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 180/374] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:30 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:30.340424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f8dfb-4545-4fd6-9288-e6a6f7f497e2 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:30.358260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:30.392672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:30.393306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.393587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.394147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.405935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:30.429455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:30.533520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:30.535798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:30.546069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7d5fc-5bf1-4267-a475-5c47f7b6596e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:30.582994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f8a1f-5181-4660-80ed-a655aad23f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:30.583626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8a1f-5181-4660-80ed-a655aad23f75 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.583902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8a1f-5181-4660-80ed-a655aad23f75 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.584172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8a1f-5181-4660-80ed-a655aad23f75 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.585263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f8a1f-5181-4660-80ed-a655aad23f75 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:30.586757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f8a1f-5181-4660-80ed-a655aad23f75 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:30.588642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 181/375] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:30 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:30.589832 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf207e5-b270-4c8d-a3fc-8e4c21cc73c3 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.590113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf207e5-b270-4c8d-a3fc-8e4c21cc73c3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.590380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf207e5-b270-4c8d-a3fc-8e4c21cc73c3 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.591585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 182/376] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:30.597452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43b28-014b-4bb4-9e52-7663742ef530 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.597737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43b28-014b-4bb4-9e52-7663742ef530 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.598031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43b28-014b-4bb4-9e52-7663742ef530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.621705 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-9aa43b28-014b-4bb4-9e52-7663742ef530 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:30.622945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9aa43b28-014b-4bb4-9e52-7663742ef530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:30.710335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 183/377] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:30.711208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43b28-014b-4bb4-9e52-7663742ef530 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:30.712010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 195/378] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:41:30.718902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:30.761501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:30.781563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:30.830840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43b28-014b-4bb4-9e52-7663742ef530 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:30.895796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb61f8c-a14b-4292-b966-e32b1e76ea41 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:30.906905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:30.907529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.907809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.908723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.909592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:30.910000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:30.911454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:30.922462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 184/379] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:30 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:30.967230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:30.967955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:30.968310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:30.968593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] PATH_INFO: `/v3/projects/17c0369ad3194a3b8c268accfe2208d0` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:30.969786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=17c0369ad3194a3b8c268accfe2208d0)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:30.971996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:30.974036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 196/380] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:30 2022] GET /identity/v3/projects/17c0369ad3194a3b8c268accfe2208d0 => generated 344 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:30.979987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:31.108046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:31.108613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab46d3-4564-434d-8612-de1fb4d757ce None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:31.118151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:31.150872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:31.151509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.151783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.152050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] PATH_INFO: `/v3/groups/4e96e50dac3f4e27b17f4739b9690c78` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.159359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] RBAC: Authorizing `identity:get_group(group_id=4e96e50dac3f4e27b17f4739b9690c78)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:31.160919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:31.163023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 185/381] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:30 2022] GET /identity/v3/groups/4e96e50dac3f4e27b17f4739b9690c78 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:31.168519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:31.261820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b6322-9363-4682-b2d5-c36bf9502cf9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:31.272267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:31.316032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:31.322803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.324533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.324947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.350850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:31.357347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf260ca-3bab-41f0-bcb8-8a5db1148297 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:31.360636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:31.361709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:31.362689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:31.373849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 197/382] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:31.379947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:31.437379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:31.437619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.445841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.446279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.447086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] RBAC: Adding query filter params (group.id=4e96e50dac3f4e27b17f4739b9690c78, role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=17c0369ad3194a3b8c268accfe2208d0) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:31.447402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:31.448885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:31.455976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 186/383] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:31 2022] GET /identity/v3/role_assignments?group.id=4e96e50dac3f4e27b17f4739b9690c78&scope.project.id=17c0369ad3194a3b8c268accfe2208d0&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 260 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:31.511424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2083aa-e6e1-4109-be62-2b7f86fd661b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:31.551386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9424a9b-4636-473a-baf4-f1e2fa7428a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:31.552776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9424a9b-4636-473a-baf4-f1e2fa7428a9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.553212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9424a9b-4636-473a-baf4-f1e2fa7428a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.553624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9424a9b-4636-473a-baf4-f1e2fa7428a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.554548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9424a9b-4636-473a-baf4-f1e2fa7428a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:31.555026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9424a9b-4636-473a-baf4-f1e2fa7428a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:31.556642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9424a9b-4636-473a-baf4-f1e2fa7428a9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:31.567544 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 198/384] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:31.574131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:31.626702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:31.696799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ed61-f274-4c88-a24d-631baf7fe390 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.697415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ed61-f274-4c88-a24d-631baf7fe390 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.698092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ed61-f274-4c88-a24d-631baf7fe390 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.699707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 199/385] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:31.701858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8a65f-edca-460a-b6fb-5dc84cfce1fb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:31.705508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1c381-8bb1-42f1-b96e-7623d028fe5f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.706018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1c381-8bb1-42f1-b96e-7623d028fe5f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.706429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1c381-8bb1-42f1-b96e-7623d028fe5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.728759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-22a1c381-8bb1-42f1-b96e-7623d028fe5f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:31.730415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-22a1c381-8bb1-42f1-b96e-7623d028fe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:31.739328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:31.740005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.740286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.740561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.741746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:31.743612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:31.745547 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 187/386] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:31 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:31.750723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:31.769444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:31.860642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 200/387] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:31.918320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a94d5-e92a-4048-8685-34f3e34a2243 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:31.919732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:31.958928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:31.959554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:31.959881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:31.960154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:31.961313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:31.962831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:31.964465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 188/388] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:31 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:31.970302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1c381-8bb1-42f1-b96e-7623d028fe5f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:32.088317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1c381-8bb1-42f1-b96e-7623d028fe5f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:32.135672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:32.136344 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:32.136616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:32.136882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/groups/6349e165b3214b48b3acf439923f3505/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:32.138807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bca7c40ebb9145f49bb3f6af4bb61457, group_id=6349e165b3214b48b3acf439923f3505, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:32.140223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:32.261374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:32.358423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 201/389] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Sep 1 13:41:31 2022] PUT /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/groups/6349e165b3214b48b3acf439923f3505/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:32.392854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6b0d8-5330-4436-a430-d15fef7fcb83 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:32.435478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:32.436134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:32.436419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:32.436693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:32.448892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:32.455990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:32.456866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:32.457669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 189/390] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:32.464717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:32.505558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:32.623092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba64fb-76e8-416f-8a90-846e19aaeec3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:32.661557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889f49b-2bc9-4191-8d01-00958dd50192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:32.662207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889f49b-2bc9-4191-8d01-00958dd50192 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:32.662485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889f49b-2bc9-4191-8d01-00958dd50192 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:32.662758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889f49b-2bc9-4191-8d01-00958dd50192 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:32.663430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889f49b-2bc9-4191-8d01-00958dd50192 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:32.663731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889f49b-2bc9-4191-8d01-00958dd50192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:32.665099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889f49b-2bc9-4191-8d01-00958dd50192 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:32.675040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 202/391] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:32.680603 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:32.746665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:32.757097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968118-b5ae-43bf-ae5b-e2d93a43829f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:32.757381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968118-b5ae-43bf-ae5b-e2d93a43829f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:32.759799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968118-b5ae-43bf-ae5b-e2d93a43829f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:32.759799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 203/392] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:32.770341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6f805-07da-4a88-ae94-5f33ff5e6364 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:32.770634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6f805-07da-4a88-ae94-5f33ff5e6364 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:32.770898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6f805-07da-4a88-ae94-5f33ff5e6364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:32.802821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-bfa6f805-07da-4a88-ae94-5f33ff5e6364 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:32.804025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bfa6f805-07da-4a88-ae94-5f33ff5e6364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:32.881645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f76e3-a304-4d9e-9d9a-5fef57cee6dd None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:32.952837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:32.953476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:32.953731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:32.954027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:32.955845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:32.957325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:32.961517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 190/393] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:32 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:32.969034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:32.972054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 204/394] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:33.117011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:33.121755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6c5d9-11a3-47da-8da4-4fc444d66b6b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:33.173014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:33.174214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.174555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.174887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.178446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:33.179950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:33.181975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 191/395] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:32 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:33.187815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6f805-07da-4a88-ae94-5f33ff5e6364 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:33.209225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:33.339030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6f805-07da-4a88-ae94-5f33ff5e6364 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:33.350831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:33.381727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16079623-80e7-434a-8ce2-1bb118a90a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:33.382563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079623-80e7-434a-8ce2-1bb118a90a2b None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.382934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079623-80e7-434a-8ce2-1bb118a90a2b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.383301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079623-80e7-434a-8ce2-1bb118a90a2b None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/groups/6349e165b3214b48b3acf439923f3505/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.387202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16079623-80e7-434a-8ce2-1bb118a90a2b None admin] RBAC: Authorizing `identity:create_grant(project_id=bca7c40ebb9145f49bb3f6af4bb61457, group_id=6349e165b3214b48b3acf439923f3505, role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:33.388733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16079623-80e7-434a-8ce2-1bb118a90a2b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:33.477582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2462ec-cd45-409e-9a83-c308587d9a68 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:33.490243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 205/396] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Sep 1 13:41:33 2022] PUT /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457/groups/6349e165b3214b48b3acf439923f3505/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:33.527403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:33.528254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.528645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.528985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.542400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:33.547763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:33.548690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:33.549505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 192/397] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:33.553928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2811f9-37cf-47d2-ba58-aa957044faf1 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.554321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2811f9-37cf-47d2-ba58-aa957044faf1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.554684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2811f9-37cf-47d2-ba58-aa957044faf1 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.554897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:33.556520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 206/398] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:33.562340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aa271-b1da-46d7-a7d3-a6e6127513a3 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.562716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aa271-b1da-46d7-a7d3-a6e6127513a3 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.563087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aa271-b1da-46d7-a7d3-a6e6127513a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.582422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3e8aa271-b1da-46d7-a7d3-a6e6127513a3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:33.583779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3e8aa271-b1da-46d7-a7d3-a6e6127513a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:33.611904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:33.702164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:33.711194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 207/399] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:33.741693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a12b71-4260-4ec1-a84d-049eee9d6fcb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:33.778321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b98a3e-c257-4fd2-9543-0ee04684f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:33.778930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b98a3e-c257-4fd2-9543-0ee04684f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.779206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b98a3e-c257-4fd2-9543-0ee04684f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.779468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b98a3e-c257-4fd2-9543-0ee04684f8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.780138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b98a3e-c257-4fd2-9543-0ee04684f8a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:33.780433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b98a3e-c257-4fd2-9543-0ee04684f8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:33.781897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b98a3e-c257-4fd2-9543-0ee04684f8a6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:33.791273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 193/400] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:33.799607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8aa271-b1da-46d7-a7d3-a6e6127513a3 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:33.925918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8aa271-b1da-46d7-a7d3-a6e6127513a3 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:33.969610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e84f4-64a0-472c-86dc-24264bac20b3 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.970165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e84f4-64a0-472c-86dc-24264bac20b3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.970273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e84f4-64a0-472c-86dc-24264bac20b3 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.973877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 194/401] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:33.973986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:33.974533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.974804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.975065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.980460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:33.985967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a469c15-9e48-4874-8f47-e16843f21566 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:33.986088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:33.986177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a469c15-9e48-4874-8f47-e16843f21566 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:33.986177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a469c15-9e48-4874-8f47-e16843f21566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:33.986302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 208/402] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:33 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:33.986302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.014978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6a469c15-9e48-4874-8f47-e16843f21566 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:34.016114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6a469c15-9e48-4874-8f47-e16843f21566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:34.064454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:34.113407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1afa6-2acd-4891-b089-97870c4388c9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:34.124693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 195/403] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:34.126107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a469c15-9e48-4874-8f47-e16843f21566 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.150740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:34.153378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:34.154113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:34.154387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:34.154657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:34.155796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:34.157288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:34.159844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 209/404] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:33 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:34.170717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.242533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a469c15-9e48-4874-8f47-e16843f21566 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:34.289747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:34.290373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:34.290647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:34.290908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:34.315297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:34.320306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:34.321839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:34.323322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:34.330040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 196/405] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:34.331958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.353757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03354af-273f-4d63-8bcb-8f88759128e1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:34.391639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:34.392861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:34.393143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:34.393412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/groups/6349e165b3214b48b3acf439923f3505/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:34.395445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d0b552698c54af99ff7c548b8d1caeb, group_id=6349e165b3214b48b3acf439923f3505, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:34.396926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:34.400185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:34.456494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:34.479195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56a242-e804-41f8-a9d8-b57ef998340d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:34.506493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 210/406] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Sep 1 13:41:34 2022] PUT /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/groups/6349e165b3214b48b3acf439923f3505/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:34.524335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:34.524778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.525060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:34.525397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:34.525672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:34.526423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:34.526728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:34.528147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:34.537306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 197/407] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:34.543199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.652495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e499078-f7c8-4844-9ddc-b29a69a7af3f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:34.680781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881161f-0145-4cfa-9f2f-e928e5950d27 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:34.689868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:34.690555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:34.690836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:34.691107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:34.692263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:34.693718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:34.699888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 198/408] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:34 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:34.701307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.714611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:34.766560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:34.768030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:34.768518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:34.769021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:34.799256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:34.812467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:34.813396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:34.814785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:34.815373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a557d8-9c5f-4097-82e7-64660c3526c2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:34.815776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 211/409] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:34.826000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.849452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:34.850269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:34.850608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:34.851046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:34.852437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:34.854042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:34.856157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 199/410] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:34 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:34.862381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:34.933383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:34.950232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:34.974074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9e6d0-4230-4dab-9119-aa40c3eb9343 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:35.009130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0cb24-1a7d-4571-bc78-3fc610451c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:35.009869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0cb24-1a7d-4571-bc78-3fc610451c84 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.010152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0cb24-1a7d-4571-bc78-3fc610451c84 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.010437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0cb24-1a7d-4571-bc78-3fc610451c84 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/groups/6349e165b3214b48b3acf439923f3505/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.012639 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0cb24-1a7d-4571-bc78-3fc610451c84 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d0b552698c54af99ff7c548b8d1caeb, group_id=6349e165b3214b48b3acf439923f3505, role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:35.014210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0cb24-1a7d-4571-bc78-3fc610451c84 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:35.044173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d745e-a9ea-4e4e-a0d5-3b0070a4527d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:35.091046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:35.092056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.092542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.093024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.094367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:35.094959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:35.097577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:35.112573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 212/411] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:35.121715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:35.178181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 200/412] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Sep 1 13:41:34 2022] PUT /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb/groups/6349e165b3214b48b3acf439923f3505/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:41:35.194810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d88f-6236-4658-97a3-39433bd39649 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.195113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d88f-6236-4658-97a3-39433bd39649 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.195488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d88f-6236-4658-97a3-39433bd39649 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.196993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 201/413] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:35.209083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a881a-5688-40ef-a55f-81cea22ab1c7 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.209347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a881a-5688-40ef-a55f-81cea22ab1c7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.209624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a881a-5688-40ef-a55f-81cea22ab1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.250214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c62a881a-5688-40ef-a55f-81cea22ab1c7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:35.251975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c62a881a-5688-40ef-a55f-81cea22ab1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:35.320760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d19f5-5e99-450a-adf9-ead7e90a7dba None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:35.382268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:35.392633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:35.393624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.396752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.397228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] PATH_INFO: `/v3/groups/4e96e50dac3f4e27b17f4739b9690c78` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.401085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] RBAC: Authorizing `identity:get_group(group_id=4e96e50dac3f4e27b17f4739b9690c78)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:35.403867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:35.406707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 213/414] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:35 2022] GET /identity/v3/groups/4e96e50dac3f4e27b17f4739b9690c78 => generated 229 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:35.428185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:35.441512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 202/415] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:35.544958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842cd6d-a5d9-4e01-98bf-ae1800c3d585 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:35.580591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:35.581222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.581499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.581790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] PATH_INFO: `/v3/projects/17c0369ad3194a3b8c268accfe2208d0` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.582926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] RBAC: Authorizing `identity:get_project(project_id=17c0369ad3194a3b8c268accfe2208d0)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:35.584347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:35.586313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 214/416] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:35 2022] GET /identity/v3/projects/17c0369ad3194a3b8c268accfe2208d0 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:35.591862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a881a-5688-40ef-a55f-81cea22ab1c7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:35.683894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:35.720101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a881a-5688-40ef-a55f-81cea22ab1c7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:35.723017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:35.766300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:35.767384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.767704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.768205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] PATH_INFO: `/v3/projects/17c0369ad3194a3b8c268accfe2208d0/groups/4e96e50dac3f4e27b17f4739b9690c78/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.770396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] RBAC: Authorizing `identity:create_grant(project_id=17c0369ad3194a3b8c268accfe2208d0, group_id=4e96e50dac3f4e27b17f4739b9690c78, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:35.771939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:35.805975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:35.853576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f88f83-f3a2-4eab-87e6-01d0ca4e72a7 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:35.872566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 203/417] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Sep 1 13:41:35 2022] PUT /identity/v3/projects/17c0369ad3194a3b8c268accfe2208d0/groups/4e96e50dac3f4e27b17f4739b9690c78/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:35.893193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:35.893875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:35.894150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:35.894415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:35.906214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:35.911700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:35.913041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:35.914751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 215/418] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:35.919004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:36.049484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7456a4-df17-4bd3-8dd9-3cec1facdb8c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:36.097728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:36.109133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:36.109746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.112814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.112814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.112814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:36.112814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:36.113999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:36.125090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 204/419] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:36.131065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:36.243326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88f63-32e0-43b8-bbed-3d32cf02edb5 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:36.257988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:36.284687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e19db-bf06-4efd-b675-0cd484d6e80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:36.285315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e19db-bf06-4efd-b675-0cd484d6e80f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.285591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e19db-bf06-4efd-b675-0cd484d6e80f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.285887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e19db-bf06-4efd-b675-0cd484d6e80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.357111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e19db-bf06-4efd-b675-0cd484d6e80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:36.358735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e19db-bf06-4efd-b675-0cd484d6e80f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:36.462286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 216/420] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:36.470089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:36.584807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f5548-0e08-4e9e-b4c8-5b619bd12129 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.585152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f5548-0e08-4e9e-b4c8-5b619bd12129 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.585365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f5548-0e08-4e9e-b4c8-5b619bd12129 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.586230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5addb2b-79ac-4e01-97b9-f7a598e0b57b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:36.586987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 217/421] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:36.592082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23abe8-eb56-44b4-9fff-88f473692623 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.592377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23abe8-eb56-44b4-9fff-88f473692623 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.592642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23abe8-eb56-44b4-9fff-88f473692623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.612578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8d23abe8-eb56-44b4-9fff-88f473692623 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:36.613696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8d23abe8-eb56-44b4-9fff-88f473692623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:36.629274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:36.633675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:36.634572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.635056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.635475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.638013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:36.639748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:36.641756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 205/422] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:36 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:36.654127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:36.697798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 218/423] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:36.748615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:36.796102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6131d-3afb-437e-af25-79aeb4f5b8b5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:36.837491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49bd4c-e19a-468d-9a76-121eb38305b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:36.838441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49bd4c-e19a-468d-9a76-121eb38305b5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.838974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49bd4c-e19a-468d-9a76-121eb38305b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.839425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49bd4c-e19a-468d-9a76-121eb38305b5 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.841960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49bd4c-e19a-468d-9a76-121eb38305b5 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:36.843681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49bd4c-e19a-468d-9a76-121eb38305b5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:36.845518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 206/424] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:36 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:36.854912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23abe8-eb56-44b4-9fff-88f473692623 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:36.926284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61a1e-4e97-4716-897c-ee9d49464c62 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.926561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61a1e-4e97-4716-897c-ee9d49464c62 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.926834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61a1e-4e97-4716-897c-ee9d49464c62 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.929468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 207/425] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:36.933438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115208d9-2641-4b3e-99f0-28454d699705 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:36.933714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115208d9-2641-4b3e-99f0-28454d699705 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:36.934024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115208d9-2641-4b3e-99f0-28454d699705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:36.974992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-115208d9-2641-4b3e-99f0-28454d699705 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:36.976456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-115208d9-2641-4b3e-99f0-28454d699705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:36.995465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23abe8-eb56-44b4-9fff-88f473692623 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:37.044965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:37.046709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:37.047176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:37.047626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:37.049235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:37.050048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:37.051784 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:37.053911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 208/426] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:37.059335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 219/427] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:36 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:37.059485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115208d9-2641-4b3e-99f0-28454d699705 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:37.169324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:37.227144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115208d9-2641-4b3e-99f0-28454d699705 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:37.270215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:37.270846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:37.271122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:37.271450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:37.285876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:37.291016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:37.291900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:37.292827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 209/428] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:37.298430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:37.313160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:37.426856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:37.428822 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ed872-e515-44de-bdc5-5ef782ed76b9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:37.484077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:37.487163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:37.487163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:37.487163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:37.487163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:37.487163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:37.488149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:37.497738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 220/429] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:37.503760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:37.582538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbaf55-a778-4b98-957b-b5398fc7c848 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:37.607479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:37.609737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:37.637484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:37.638160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:37.638443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:37.638720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:37.655918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf2de7-d617-4e4e-8339-5f75e1c565f5 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:37.661307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:37.666581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:37.667498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 13:41:37.668293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:37.669103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 210/430] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:37.678145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:37.712206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383352de-3a87-4eed-ba50-cda2bc31f907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:37.713067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383352de-3a87-4eed-ba50-cda2bc31f907 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:37.714062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383352de-3a87-4eed-ba50-cda2bc31f907 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:37.714515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383352de-3a87-4eed-ba50-cda2bc31f907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:37.808725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383352de-3a87-4eed-ba50-cda2bc31f907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:37.809666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60def4a5-da0a-42c8-8dd2-56b6c21d36f5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:37.811954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383352de-3a87-4eed-ba50-cda2bc31f907 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:37.849436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:37.850346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:37.850768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:37.851187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:37.852055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] RBAC: Adding query filter params (name=member) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:37.852606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:37.854272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:37.864682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 211/431] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:41:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:37.872108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:37.891522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:37.912634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:37.933473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 221/432] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:37.935164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52040e3-a198-43c3-9891-9c65487e21e4 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:37.935450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52040e3-a198-43c3-9891-9c65487e21e4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:37.935720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52040e3-a198-43c3-9891-9c65487e21e4 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:37.937106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 222/433] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:37.941708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52040e3-a198-43c3-9891-9c65487e21e4 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:37.991914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78998a-8dcc-4b08-9acb-43e634de63ba None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.029411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.030183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.030458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.030849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.051580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52040e3-a198-43c3-9891-9c65487e21e4 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.093710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dd4be-0c54-4544-bfae-5700d2c6ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.094329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dd4be-0c54-4544-bfae-5700d2c6ae72 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.094601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dd4be-0c54-4544-bfae-5700d2c6ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.094910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dd4be-0c54-4544-bfae-5700d2c6ae72 None admin] PATH_INFO: `/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.096133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dd4be-0c54-4544-bfae-5700d2c6ae72 None admin] RBAC: Authorizing `identity:get_project(project_id=bca7c40ebb9145f49bb3f6af4bb61457)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:38.097492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dd4be-0c54-4544-bfae-5700d2c6ae72 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:38.103795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 223/434] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:37 2022] GET /identity/v3/projects/bca7c40ebb9145f49bb3f6af4bb61457 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:38.103795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c419a-5b69-4174-9885-1eadbd86c78f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.103795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c419a-5b69-4174-9885-1eadbd86c78f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.103795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c419a-5b69-4174-9885-1eadbd86c78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.109414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:38.110967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:38.120035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-588c419a-5b69-4174-9885-1eadbd86c78f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:38.121222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-588c419a-5b69-4174-9885-1eadbd86c78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:38.151927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:38.154483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:38.198065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 224/435] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:38.198065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c419a-5b69-4174-9885-1eadbd86c78f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.240148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 212/436] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:38.241386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.325735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c419a-5b69-4174-9885-1eadbd86c78f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.357275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d468d-2707-49f1-92fe-d9e1b3f087a7 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.367994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e558-bada-4005-a8f0-5ed656c9ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.369358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e558-bada-4005-a8f0-5ed656c9ae65 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.369808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e558-bada-4005-a8f0-5ed656c9ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.370255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e558-bada-4005-a8f0-5ed656c9ae65 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.372230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e558-bada-4005-a8f0-5ed656c9ae65 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:38.374286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e558-bada-4005-a8f0-5ed656c9ae65 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:38.376399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 225/437] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:38.378341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f93b8-ccfd-4552-934b-a72a96ca36f7 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.378619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f93b8-ccfd-4552-934b-a72a96ca36f7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.378886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f93b8-ccfd-4552-934b-a72a96ca36f7 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.380259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 226/438] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:38.382878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f93b8-ccfd-4552-934b-a72a96ca36f7 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.393314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4683954-78a5-4f73-9d96-56f6891fef8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.393974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4683954-78a5-4f73-9d96-56f6891fef8c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.394248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4683954-78a5-4f73-9d96-56f6891fef8c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.394521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4683954-78a5-4f73-9d96-56f6891fef8c None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.395628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4683954-78a5-4f73-9d96-56f6891fef8c None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:38.397086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4683954-78a5-4f73-9d96-56f6891fef8c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:38.402980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 213/439] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:38.402980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acdcc9-5e72-48d7-96a5-099e548291aa None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.402980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acdcc9-5e72-48d7-96a5-099e548291aa None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.402980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acdcc9-5e72-48d7-96a5-099e548291aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.408153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:38.419031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-33acdcc9-5e72-48d7-96a5-099e548291aa None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:38.420209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-33acdcc9-5e72-48d7-96a5-099e548291aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:38.428241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:38.488182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 214/440] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:38.488941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acdcc9-5e72-48d7-96a5-099e548291aa None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.494476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f93b8-ccfd-4552-934b-a72a96ca36f7 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.529508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.530144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.530425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.530696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.531426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:38.531734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:38.533956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:38.549802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 227/441] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=bca7c40ebb9145f49bb3f6af4bb61457&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:38.549802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.619031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acdcc9-5e72-48d7-96a5-099e548291aa None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.663129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.664157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.664268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.664472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.665708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:38.667267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:38.669546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 215/442] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:38.678426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.690587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:38.691642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:38.694168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fb401-3c02-48f3-a6dc-ccbe93e8e095 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.732532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.733258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.733504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.733823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.751708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:38.758690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:38.759556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:38.760391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 228/443] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:38.765988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.808135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696dbcd-18be-4cac-9a2a-ced0e40d40bf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.851323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.851529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.851688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.851911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.852750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=9f2bf5a138d54952a913a69eb59fa144, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:38.853057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:38.854600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:38.862800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 216/444] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=9f2bf5a138d54952a913a69eb59fa144 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:38.863631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.895535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30a2db-ec6f-4957-a38d-6d1b0e3480af None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:38.939989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:38.940636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:38.940916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:38.941186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:38.944188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:38.960080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:38.966036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:38.967014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:38.967791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 229/445] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:38.975456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:38.999879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbcef0-c926-4634-91c4-d40a9c15e2c8 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.011771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.042185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.042806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.043085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.043359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.044063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:39.044358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.045884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.056644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 217/446] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:39.062443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.109507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d18dc-0560-4868-8bd0-b6f552e7ab22 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.152009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.152984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.153262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.153531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.154548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:39.155140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.156690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.167283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 230/447] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:39.174714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.181051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131053d-9c12-43cf-843d-b581e29f03d3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.219072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.231401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.233472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.233472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.233472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.270416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.290251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edd8c9-750c-459b-b8fa-a2873bbf46fe None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.303291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.304742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.324891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.325500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.325792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.326074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.393995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.395438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.402482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 218/448] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:41:39.410289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.436055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.475893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.495936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 231/449] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 13:41:39.502191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.508102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf5a6-e4be-4460-97eb-51c22092aa81 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.542354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.542913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.543190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.543464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] PATH_INFO: `/v3/projects/7d0b552698c54af99ff7c548b8d1caeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.544587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] RBAC: Authorizing `identity:get_project(project_id=7d0b552698c54af99ff7c548b8d1caeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.546143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.547765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 219/450] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/projects/7d0b552698c54af99ff7c548b8d1caeb => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:39.553352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.603528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719ef20-53f9-46a9-974c-73a5d4b16c5a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.637502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.638152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.638427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.638695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] PATH_INFO: `/v3/projects/17c0369ad3194a3b8c268accfe2208d0` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.639821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] RBAC: Authorizing `identity:get_project(project_id=17c0369ad3194a3b8c268accfe2208d0)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.641254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.643539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 232/451] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/projects/17c0369ad3194a3b8c268accfe2208d0 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:39.647203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.649081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.652933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c8697-34ce-4ae3-a796-40fb80d93988 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.685241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.685891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.686186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.686459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] PATH_INFO: `/v3/groups/6349e165b3214b48b3acf439923f3505` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.687594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] RBAC: Authorizing `identity:get_group(group_id=6349e165b3214b48b3acf439923f3505)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.689648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.691658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 220/452] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/groups/6349e165b3214b48b3acf439923f3505 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:39.697553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.701195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.749934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919ac00-23b7-41be-83f5-d9b91bbd6b06 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.783840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.784466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.784741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.785013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] PATH_INFO: `/v3/groups/4e96e50dac3f4e27b17f4739b9690c78` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.786179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] RBAC: Authorizing `identity:get_group(group_id=4e96e50dac3f4e27b17f4739b9690c78)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.787601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.789157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 233/453] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/groups/4e96e50dac3f4e27b17f4739b9690c78 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:39.795071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:39.795969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86495c4a-f6cc-4bd3-9f1d-51635496bcb3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.830262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230be92f-7c3f-47d5-95a1-bafa65f2623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.830858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230be92f-7c3f-47d5-95a1-bafa65f2623a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.831160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230be92f-7c3f-47d5-95a1-bafa65f2623a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.831506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230be92f-7c3f-47d5-95a1-bafa65f2623a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.832137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230be92f-7c3f-47d5-95a1-bafa65f2623a None admin] RBAC: Adding query filter params (group.id=6349e165b3214b48b3acf439923f3505, role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=7d0b552698c54af99ff7c548b8d1caeb) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:39.832441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230be92f-7c3f-47d5-95a1-bafa65f2623a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.833895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230be92f-7c3f-47d5-95a1-bafa65f2623a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.840148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 221/454] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/role_assignments?group.id=6349e165b3214b48b3acf439923f3505&scope.project.id=7d0b552698c54af99ff7c548b8d1caeb&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:39.900846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059856-b1bb-48c0-a086-ca8f4ae86e44 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:39.919345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:39.936923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e1077-ea21-40b5-84b3-f86b4cdf52cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:39.937539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1077-ea21-40b5-84b3-f86b4cdf52cf None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:39.937840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1077-ea21-40b5-84b3-f86b4cdf52cf None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:39.938112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1077-ea21-40b5-84b3-f86b4cdf52cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:39.939450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1077-ea21-40b5-84b3-f86b4cdf52cf None admin] RBAC: Adding query filter params (group.id=4e96e50dac3f4e27b17f4739b9690c78, role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=17c0369ad3194a3b8c268accfe2208d0) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:39.939754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1077-ea21-40b5-84b3-f86b4cdf52cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:39.941188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1077-ea21-40b5-84b3-f86b4cdf52cf None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:39.947843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 234/455] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Sep 1 13:41:39 2022] GET /identity/v3/role_assignments?group.id=4e96e50dac3f4e27b17f4739b9690c78&scope.project.id=17c0369ad3194a3b8c268accfe2208d0&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:42.293942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bda8a-c288-42fe-ba7c-07b5461d88f6 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:42.294977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bda8a-c288-42fe-ba7c-07b5461d88f6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:42.295463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bda8a-c288-42fe-ba7c-07b5461d88f6 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:42.297309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 222/456] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:42.303056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54442b04-5770-4260-a90c-d1be1fe6a328 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:42.303487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54442b04-5770-4260-a90c-d1be1fe6a328 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:42.303677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54442b04-5770-4260-a90c-d1be1fe6a328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:42.326522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-54442b04-5770-4260-a90c-d1be1fe6a328 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:42.327730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-54442b04-5770-4260-a90c-d1be1fe6a328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:42.422568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 235/457] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:42.486515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0b99-5351-459e-9130-f52f74d5d649 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:42.487138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0b99-5351-459e-9130-f52f74d5d649 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:42.487610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0b99-5351-459e-9130-f52f74d5d649 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:42.489373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 223/458] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:42.495020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a2ba4-a3e5-4652-8cf9-c3bfab0015b9 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:42.495304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a2ba4-a3e5-4652-8cf9-c3bfab0015b9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:42.495826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a2ba4-a3e5-4652-8cf9-c3bfab0015b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:42.514291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-244a2ba4-a3e5-4652-8cf9-c3bfab0015b9 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:42.515740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-244a2ba4-a3e5-4652-8cf9-c3bfab0015b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:42.586490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 236/459] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:42.778441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0b99-5351-459e-9130-f52f74d5d649 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:42.808743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:42.857272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5091fd-9a1c-4c8e-bc19-cd87100eb7e9 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:42.857899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5091fd-9a1c-4c8e-bc19-cd87100eb7e9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:42.858392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5091fd-9a1c-4c8e-bc19-cd87100eb7e9 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:42.863498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 237/460] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:42.869156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de78bc-8cfd-4260-b6ed-e4137f85e39f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:42.869788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de78bc-8cfd-4260-b6ed-e4137f85e39f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:42.870262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de78bc-8cfd-4260-b6ed-e4137f85e39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:42.899084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d3de78bc-8cfd-4260-b6ed-e4137f85e39f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:42.900744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d3de78bc-8cfd-4260-b6ed-e4137f85e39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:42.911142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0b99-5351-459e-9130-f52f74d5d649 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:42.940790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:42.968432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:42.969077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:42.969358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:42.969631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:42.981078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 238/461] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:42.982720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:42.983588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de78bc-8cfd-4260-b6ed-e4137f85e39f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:42.986326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:43.016843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:43.021264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:43.022153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:43.022153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:43.022153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:43.022153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 224/462] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:43.026929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:43.182213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:43.201959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c91be-9453-4305-8f52-711716b4b672 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:43.215457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de78bc-8cfd-4260-b6ed-e4137f85e39f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:43.239198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:43.240071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.240499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.240910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.241693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:43.242155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:43.244037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.262900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 225/463] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:43.269184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:43.298086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:43.298715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.299207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.299676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.314390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:43.316245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:43.336435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:43.337375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:43.338275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:43.338275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:43.338275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:43.338435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 239/464] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:43.339308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:43.367954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:43.391658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee456790-dfd8-4119-97e6-88ac07f5cabe None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:43.427115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3f139-c70c-4ef2-a6ee-954fbedc2612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:43.427732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3f139-c70c-4ef2-a6ee-954fbedc2612 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.428018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3f139-c70c-4ef2-a6ee-954fbedc2612 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.428284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3f139-c70c-4ef2-a6ee-954fbedc2612 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.428932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3f139-c70c-4ef2-a6ee-954fbedc2612 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:43.430437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3f139-c70c-4ef2-a6ee-954fbedc2612 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.443246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:43.448629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e56df-050b-4526-910d-e1540fa0900e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:43.482887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e630c6-cee4-46e1-b027-9c9a957ea9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:43.483512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e630c6-cee4-46e1-b027-9c9a957ea9f9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.483790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e630c6-cee4-46e1-b027-9c9a957ea9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.484062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e630c6-cee4-46e1-b027-9c9a957ea9f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.494336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e630c6-cee4-46e1-b027-9c9a957ea9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:43.495761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e630c6-cee4-46e1-b027-9c9a957ea9f9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-d1e630c6-cee4-46e1-b027-9c9a957ea9f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:43.513366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:43.514582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:43.515479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:43.515479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:43.515479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:43.515479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 240/465] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:43.516375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d167c-3025-471f-99da-80d5b5052a8b None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.516661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d167c-3025-471f-99da-80d5b5052a8b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.517570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d167c-3025-471f-99da-80d5b5052a8b None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.521982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 241/466] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:43.521982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d167c-3025-471f-99da-80d5b5052a8b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:43.553068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 226/467] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:43 2022] POST /identity/v3/users => generated 246 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:43.554657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161019-9f61-4306-885b-986e51aa8bc7 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.555126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161019-9f61-4306-885b-986e51aa8bc7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.555553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161019-9f61-4306-885b-986e51aa8bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.578659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-77161019-9f61-4306-885b-986e51aa8bc7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:43.581992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-77161019-9f61-4306-885b-986e51aa8bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:43.634947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:43.666222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d167c-3025-471f-99da-80d5b5052a8b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:43.692685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 227/468] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:43.694444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161019-9f61-4306-885b-986e51aa8bc7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:43.719079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:43.720076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.721335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.721828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.722913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:43.723215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:43.724584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.741354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 242/469] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:43.747688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:43.839023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161019-9f61-4306-885b-986e51aa8bc7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:43.867083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:43.874582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0163d-3e96-4d70-961b-ac7d41fd70aa None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:43.878110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:43.884449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.884449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.884449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.884449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:43.884449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:43.884449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.895893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 228/470] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:43.904092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:43.923676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e486bfe-8eff-41d5-9037-b9fc1d80a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:43.924293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486bfe-8eff-41d5-9037-b9fc1d80a474 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:43.924569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486bfe-8eff-41d5-9037-b9fc1d80a474 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:43.924834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486bfe-8eff-41d5-9037-b9fc1d80a474 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:43.925483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e486bfe-8eff-41d5-9037-b9fc1d80a474 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:43.926961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e486bfe-8eff-41d5-9037-b9fc1d80a474 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:43.936932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:44.018357 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2651-ee26-4b1b-bd47-40c6911fbbc9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:44.055807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4db1fa-0924-4648-91d6-650348750b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:44.056466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4db1fa-0924-4648-91d6-650348750b1f None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:44.056747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4db1fa-0924-4648-91d6-650348750b1f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:44.057020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4db1fa-0924-4648-91d6-650348750b1f None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:44.057670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4db1fa-0924-4648-91d6-650348750b1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:44.059179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4db1fa-0924-4648-91d6-650348750b1f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:44.059889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 243/471] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:43 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:41:44.060969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4c27-6e95-43bf-9b1c-9a6d6c590778 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:44.061259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4c27-6e95-43bf-9b1c-9a6d6c590778 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:44.061588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4c27-6e95-43bf-9b1c-9a6d6c590778 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:44.063034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 244/472] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:44.069941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4c27-6e95-43bf-9b1c-9a6d6c590778 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:44.160911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 229/473] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:43 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:44.160911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364436ab-6c14-418c-89f4-36576266741a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:44.160911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364436ab-6c14-418c-89f4-36576266741a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:44.160911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364436ab-6c14-418c-89f4-36576266741a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:44.198066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-364436ab-6c14-418c-89f4-36576266741a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:44.201672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-364436ab-6c14-418c-89f4-36576266741a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:44.245246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4c27-6e95-43bf-9b1c-9a6d6c590778 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:44.256538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:44.299979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:44.300534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:44.300813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:44.301082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:44.303791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 230/474] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:44.313076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:44.316135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:44.332517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:44.333460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:44.334332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:44.334332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:44.334332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:44.336577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 245/475] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:44.342391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364436ab-6c14-418c-89f4-36576266741a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:44.454989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364436ab-6c14-418c-89f4-36576266741a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:44.493384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:44.494387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:44.494889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:44.495377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:44.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:44.496865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:44.498471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:44.511500 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 231/476] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:44.517948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:44.571284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:44.620361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:44.638044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4636-4755-401f-9b1a-643261b27fc1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:44.661918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:44.685075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:44.686122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:44.686595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:44.687057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:44.687891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:44.690110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:44.752444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3caf0-a1fc-4c82-883a-11ffdf482060 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:44.770618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 246/477] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:44 2022] POST /identity/v3/users => generated 249 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:44.809458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:44.810566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:44.811064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:44.811557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:44.812449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:44.814670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:44.849105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.sql.core [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 13:41:44.849105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 13:41:44.849105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [parameters: {'id': 'ddebd5646d144ba8878c8212462a238c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 13:41:44.849105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result.read() Sep 01 13:41:44.869404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:41:44.870414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application yield resource Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise self.value Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application session.commit() Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 13:41:44.871340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.session.flush() Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application compat.raise_( Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 13:41:44.872197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:41:44.873048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result.read() Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:41:44.878824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [parameters: {'id': 'ddebd5646d144ba8878c8212462a238c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:44.880826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 13:41:44.882839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:44.886764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 232/478] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Sep 1 13:41:44 2022] POST /identity/v3/roles => generated 144 bytes in 267 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:44.893789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:45.004815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:45.008864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4e07b-b8f4-4048-9a90-bec3f365493b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:45.047161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13971f8e-394e-41f4-a860-4881bac1728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:45.048901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971f8e-394e-41f4-a860-4881bac1728d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:45.049928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971f8e-394e-41f4-a860-4881bac1728d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:45.050396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971f8e-394e-41f4-a860-4881bac1728d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:45.061194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13971f8e-394e-41f4-a860-4881bac1728d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-13971f8e-394e-41f4-a860-4881bac1728d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:45.066814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:45.067700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 13:41:45.068488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:45.071085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 247/479] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:41:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:45.076999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:45.181571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:45.188253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edffb6-9ee0-4e17-a210-5554115abf19 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:45.223481 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0da43-973d-4a6e-8033-48bc15861edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:45.224413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0da43-973d-4a6e-8033-48bc15861edd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:45.224912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0da43-973d-4a6e-8033-48bc15861edd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:45.225377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0da43-973d-4a6e-8033-48bc15861edd None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:45.226956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0da43-973d-4a6e-8033-48bc15861edd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:45.227493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0da43-973d-4a6e-8033-48bc15861edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:45.229082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0da43-973d-4a6e-8033-48bc15861edd None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:45.239018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 233/480] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:46.008959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746efb-6334-439f-b6df-0fe6812373e5 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.009541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746efb-6334-439f-b6df-0fe6812373e5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.009541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746efb-6334-439f-b6df-0fe6812373e5 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.011351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 248/481] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:46.023244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce59e7d-1df1-4ee4-951e-f897d1033fd4 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.024061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce59e7d-1df1-4ee4-951e-f897d1033fd4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.024530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce59e7d-1df1-4ee4-951e-f897d1033fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.045590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3ce59e7d-1df1-4ee4-951e-f897d1033fd4 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:46.047229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3ce59e7d-1df1-4ee4-951e-f897d1033fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:46.136982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 234/482] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:46.468280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746efb-6334-439f-b6df-0fe6812373e5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:46.560332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e593ba-d6a3-4f6c-a00d-98b1d65b0e0f None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.560647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e593ba-d6a3-4f6c-a00d-98b1d65b0e0f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.560968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e593ba-d6a3-4f6c-a00d-98b1d65b0e0f None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.562673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 235/483] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:46.568817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c52d03-3260-41d6-b355-1417c2726d01 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.569105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c52d03-3260-41d6-b355-1417c2726d01 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.569382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c52d03-3260-41d6-b355-1417c2726d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.601864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-76c52d03-3260-41d6-b355-1417c2726d01 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:46.603084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-76c52d03-3260-41d6-b355-1417c2726d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:46.606166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746efb-6334-439f-b6df-0fe6812373e5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:46.649100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:46.649736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.650046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.650321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.669702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:46.675542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:46.676404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:46.677171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:46.677789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 249/484] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:46.686286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:46.719930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 236/485] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:46.741808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:46.787349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5806d5-b17e-4212-aa53-22c5f846288d None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.787679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5806d5-b17e-4212-aa53-22c5f846288d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.788001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5806d5-b17e-4212-aa53-22c5f846288d None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.789469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 237/486] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:46.795720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea73faf-3e3e-41e6-9355-ec48d2dc87ad None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.796240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea73faf-3e3e-41e6-9355-ec48d2dc87ad None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.796715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea73faf-3e3e-41e6-9355-ec48d2dc87ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.820687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5ea73faf-3e3e-41e6-9355-ec48d2dc87ad None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:46.822583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5ea73faf-3e3e-41e6-9355-ec48d2dc87ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:46.839568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:46.885717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c5146-c075-41c7-9141-924647ef4d8f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:46.927994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 238/487] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:46.946626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:46.947291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:46.947570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:46.947839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:46.948509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:46.948856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:46.950311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:46.962337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 250/488] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:46.968132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea73faf-3e3e-41e6-9355-ec48d2dc87ad None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.081583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea73faf-3e3e-41e6-9355-ec48d2dc87ad None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.117866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.118507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.118783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.119053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.129744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:47.131241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:47.140110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:47.146766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:47.147708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:47.148562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:47.148562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:47.148562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:47.148562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 239/489] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:47.154936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.180278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.291779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb34f17-a756-426d-812d-3db243aa8be0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.304492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:47.326424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75109-beac-46d9-925d-9e4630174000 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.345371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.346443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.346733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.347009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.362954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:47.365403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.366275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.366759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.367486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.368382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:47.368885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:47.370112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:47.371005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:47.371816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 240/490] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:47.373002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.373756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:47.395223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 251/491] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:47.395223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.443249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:47.487508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e870a-5f57-47ff-83e3-915eaea67af9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.531811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.533267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.533571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.533884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.535694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dcea7-772d-4e1f-b9fc-7ab03b71be88 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.544620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:47.549960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:47.550853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:47.551729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 241/492] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:47.552383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.580389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845e4a7-02a2-41f8-bd4b-347f0f1982c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.581331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e4a7-02a2-41f8-bd4b-347f0f1982c6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.581649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e4a7-02a2-41f8-bd4b-347f0f1982c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.582009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e4a7-02a2-41f8-bd4b-347f0f1982c6 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.582616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845e4a7-02a2-41f8-bd4b-347f0f1982c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:47.582885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845e4a7-02a2-41f8-bd4b-347f0f1982c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:47.584249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845e4a7-02a2-41f8-bd4b-347f0f1982c6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:47.592756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 252/493] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:47.593828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac7d1c-170d-43e9-815c-18abf8739398 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.594118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac7d1c-170d-43e9-815c-18abf8739398 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.594392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac7d1c-170d-43e9-815c-18abf8739398 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.596382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 253/494] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:47.596939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac7d1c-170d-43e9-815c-18abf8739398 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.601845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:47.664342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc091439-c9f4-4b9d-aadd-aa4d3469ee97 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.685236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:47.699585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.700212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.701615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.701615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.710263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:47.711702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:47.717009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac7d1c-170d-43e9-815c-18abf8739398 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:47.724127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:47.725013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:47.725957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 242/495] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:47.726540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.761971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515094e4-ed29-452d-9f8b-08b357646cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.762590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515094e4-ed29-452d-9f8b-08b357646cb0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.762870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515094e4-ed29-452d-9f8b-08b357646cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.763142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515094e4-ed29-452d-9f8b-08b357646cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.763822 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515094e4-ed29-452d-9f8b-08b357646cb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:47.764125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515094e4-ed29-452d-9f8b-08b357646cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:47.765495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515094e4-ed29-452d-9f8b-08b357646cb0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:47.779098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 254/496] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:47.779098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88ea58-b80e-4151-8d7f-a9af8ae70365 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.779098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88ea58-b80e-4151-8d7f-a9af8ae70365 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.779098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88ea58-b80e-4151-8d7f-a9af8ae70365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.794507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ff88ea58-b80e-4151-8d7f-a9af8ae70365 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:47.795695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ff88ea58-b80e-4151-8d7f-a9af8ae70365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:47.848429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc9064-30e7-4a7d-9666-3987041b9c8e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:47.854598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:47.862055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 255/497] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:47.863022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88ea58-b80e-4151-8d7f-a9af8ae70365 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.884113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee5183-1e48-4b4b-bd26-b3cf523f3644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:47.884743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5183-1e48-4b4b-bd26-b3cf523f3644 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.885023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5183-1e48-4b4b-bd26-b3cf523f3644 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.885299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5183-1e48-4b4b-bd26-b3cf523f3644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.896642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee5183-1e48-4b4b-bd26-b3cf523f3644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:47.898221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee5183-1e48-4b4b-bd26-b3cf523f3644 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-adee5183-1e48-4b4b-bd26-b3cf523f3644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:47.913256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:47.914223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:47.915668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:47.915668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:47.915668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:47.915668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 243/498] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:47.915881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cba66c-94c8-45aa-88fb-83aa552cbaaf None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:47.916126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cba66c-94c8-45aa-88fb-83aa552cbaaf None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:47.916398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cba66c-94c8-45aa-88fb-83aa552cbaaf None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:47.917827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 244/499] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:47.922353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cba66c-94c8-45aa-88fb-83aa552cbaaf None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:47.958062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:47.968830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88ea58-b80e-4151-8d7f-a9af8ae70365 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.006261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.006865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.007238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.007520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.008248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:48.008552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.009991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.035390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cba66c-94c8-45aa-88fb-83aa552cbaaf None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.053034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 256/500] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:48.053034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.077650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91754f-4b2b-4a54-b1c4-c607bcb16ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.078355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91754f-4b2b-4a54-b1c4-c607bcb16ad9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.078630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91754f-4b2b-4a54-b1c4-c607bcb16ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.079427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91754f-4b2b-4a54-b1c4-c607bcb16ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.090206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91754f-4b2b-4a54-b1c4-c607bcb16ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.092341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91754f-4b2b-4a54-b1c4-c607bcb16ad9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-0d91754f-4b2b-4a54-b1c4-c607bcb16ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:48.108630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:48.109513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:48.110418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:48.110418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:48.110418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:48.114112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.114622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 245/501] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:48.115996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f3828-ccf3-4160-a18b-ef797ca8762f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.116105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f3828-ccf3-4160-a18b-ef797ca8762f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.116366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f3828-ccf3-4160-a18b-ef797ca8762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.134877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5c4f3828-ccf3-4160-a18b-ef797ca8762f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:48.136070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5c4f3828-ccf3-4160-a18b-ef797ca8762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:48.160278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f81fbd-6999-4951-94d5-5836913010c3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.192894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.193505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.193806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.194089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.194734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:48.195042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.196417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.202440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 246/502] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:48.203287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f3828-ccf3-4160-a18b-ef797ca8762f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.218341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.228703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 257/503] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:48.229451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.310687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f3828-ccf3-4160-a18b-ef797ca8762f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.348122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.348865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.349151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.349423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.357089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105083-53bb-4b87-b951-cdd9714f4af1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.372833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.398998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.400056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.400624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.400902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.401546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:48.402022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.404515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.425456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 258/504] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:48.425456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.426332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.427594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.446679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.528113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3c039-1991-47a2-9174-1fbf05148546 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.537365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 247/505] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:48.538253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.563280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.563887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.564165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.564437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.574379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:48.579581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:48.580501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:48.581320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 259/506] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:48.582204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.616013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.640498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2808d-f411-42e6-99da-66f5290956a0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.654147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.675713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.676337 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.676610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.676875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.684284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.685741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61966f-79a3-4506-889b-b5f52503eba8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.685888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:48.696864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:48.697948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 248/507] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:48.698920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.726010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.726634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.726912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.727192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.734095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.736133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:48.749451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:48.750663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:41:48.751694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:48.752227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 260/508] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:48.752535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.802650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2cf4d-b3c6-419f-8a46-11ea4e74b3f9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.836939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.837880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.838150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.838410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.847506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.849606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.851117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.854315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d1e4-c5aa-42a2-9265-61724419c025 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:48.866001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:48.867113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:48.868208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:48.868208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:48.868208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:48.868208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 249/509] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:48.868208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.888493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:48.889130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:48.889410 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:48.889684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:48.896075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:48.900297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:48.901921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:48.917325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:48.918279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:48.919121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:48.919121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:48.919121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:48.919121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 261/510] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:48.919333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:48.972302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30bf4b-d609-487a-b7eb-039fefcfa4f1 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.006440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.007040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.007316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.007591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.008255 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:49.008560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.009957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.018235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 250/511] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:49.019018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.025962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1922e-4510-44f8-9088-410582bbc076 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.059867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.060468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.060748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.061019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.061804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:49.062103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.063558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.081556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 262/512] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Sep 1 13:41:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:49.082481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.086501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.150582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.163103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20681154-d82f-4ee9-bb49-7203b51c91e7 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.182593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007db8a-e373-474d-9ecf-68ec3a3eb176 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.196133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.197078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.197360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.197632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.198390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:49.198695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.200134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.217595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.218226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.218507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.218779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.219424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:49.219756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.221336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.232538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 263/513] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:49.233329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.240982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 251/514] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:49.241970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.297197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.340390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05b03-03e9-4a5d-b4cc-eb833ff32854 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.351627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122e5f8-d108-42f7-b48f-065afec543fe None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.373903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.374510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.374788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.375061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.375703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:49.376527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.377976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.385877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.386505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.386786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.387055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.389815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 264/515] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:49.390588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.397697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.399201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.406309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:49.414697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:49.420071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:49.420938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:49.420938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:49.420938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:49.420938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 252/516] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:49.420938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.499303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b20b9e-5136-4ccc-b370-82b328fd1234 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.518575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b556ed-7fcd-47c1-8636-5b4e1bce5152 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.523918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.534659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.535270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.535547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.535816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.551868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.552471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.552749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.553021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.553662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.555136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.602990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.604426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.665685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 253/517] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:41:49 2022] POST /identity/v3/users => generated 247 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:41:49.666497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.712518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 265/518] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:49.712518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.738136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.740604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.794995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26756c62-6048-4fb6-b18d-47639ae184a5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.820569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab7c9a-fd86-439a-8452-8e28f212d683 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:49.837051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.837660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.837961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.838229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.864322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:49.865048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:49.865333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:49.865956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:49.872848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.874317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:49.888712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:49.889751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 266/519] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:49.890745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:49.926264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:49.927753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:49.987829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:49.988926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:50.003049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e951-a424-4b53-a78a-ac1ad6158b1d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.035460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 254/520] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:50.036776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.044583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.045925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.046440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.047267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.048751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:50.049522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.051391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.071206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 267/521] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:50.071206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.150069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bddc4-4b0c-4ce9-82ba-0fae431bd59a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.178587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795b935-b055-4e67-b65e-67bbc78ca802 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.186815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.187454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.187732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.188035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.198876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.200343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:50.216471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:50.217429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:50.218382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:50.218382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:50.218382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:50.218560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.218560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.218560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.218560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.220187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:50.220187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.220737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.222792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 255/522] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:50.223494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.233510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:50.244980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 268/523] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:50.244980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.247043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:50.350357 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a28bf-ba45-4c1c-a80c-542eb7d6a47f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.360572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7caa5a-903e-4eba-b480-48534b63f55e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.391083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.392249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.392528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.392804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.397468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.398293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.398720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.399138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.403555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.405221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.407956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.409389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:50.420184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:50.421195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:50.422893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:50.422893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:50.422893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:50.423117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:50.423930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:41:50.424900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:50.425395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 256/524] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:50.425395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.432555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 269/525] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:50.432555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.488690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:50.498606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:50.536598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7451fd1-e141-4dfd-bf65-475a631a36c9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.537023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd290-97e2-4283-81dc-9dd70e22c628 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.571390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.571835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.571977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.572249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.572440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.572533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.572711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.572982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.573222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=19a91e958d514c0d8a61289bbe3ea63d) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:50.573520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.573610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:50.573934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.574974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.575303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.590342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 270/526] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/role_assignments?user.id=19a91e958d514c0d8a61289bbe3ea63d&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:50.590342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.603803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 257/527] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:50.604893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.707823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c6757-f0cd-49de-9160-1e72ad9805ef None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.740558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:50.750621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.751712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.751997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.756782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.756782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:50.756782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.757052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1437365-8973-4bc1-a35e-3e074a1b5c9e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.757161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.775086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 271/528] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:50.776504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:50.777330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.795005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.795898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.796243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.796589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.797396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:50.797828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.799590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.822069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 258/529] 10.222.0.61 () {56 vars in 1139 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:50.823040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.897805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e075dba-8247-46a8-adb3-632aff47e864 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.932966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.933620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.933924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.934195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.935381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16725f68-20db-4673-b51b-5cc272053784 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:50.941668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.943149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:50.955563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:50.956462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:50.957318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 272/530] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:50.958354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:50.978898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:50.979672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:50.989507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:50.989507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:50.989507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:50.990052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:51.002966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:51.005050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 259/531] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:51.006951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:51.011500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:51.046122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:51.066277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864880f-2d57-42a6-b614-076fe748aedb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:51.103016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:51.103689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.103961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.104232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.121936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74bc9e-e832-4db3-9e08-1802851b5eff None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:51.161420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:51.162512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.163095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.163589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.164478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:51.165075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:51.166963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.175620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:51.177056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.192654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 260/532] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:51.194273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:51.227551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:51.289749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 273/533] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:51.289749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:51.320199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:51.327156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f74f-0dbe-448c-9ce6-8d4cacad9446 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:51.366461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:51.367709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.368263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.368711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.369673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:51.370186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:51.371922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.388908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 261/534] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:51.390014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:51.413498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3348e5-b4cf-4e2b-a5fc-b4d71add1acd None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:51.453543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:51.454966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.455548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.455996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.456897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=42f33454f4b94b7fb54e83252ce564db) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:51.457379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:51.459203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.470329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 274/535] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:51 2022] GET /identity/v3/role_assignments?user.id=42f33454f4b94b7fb54e83252ce564db&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:51.471647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:51.502410 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:51.519437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a39bd-272f-497c-9f89-7f131329ec4c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:51.562987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:51.564184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.564653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.565113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.581252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:51.584211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:51.604803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:51.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:51.606672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:51.606672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:51.606672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:51.606672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 262/536] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:51.613550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:51.627038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:51.692128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1eed77-e7d1-4815-9491-f322d0b8de81 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:51.730549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ee77d-7e51-4376-9533-ee5d28c78ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:51.731557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ee77d-7e51-4376-9533-ee5d28c78ca3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.732039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ee77d-7e51-4376-9533-ee5d28c78ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.732545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ee77d-7e51-4376-9533-ee5d28c78ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.733604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ee77d-7e51-4376-9533-ee5d28c78ca3 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=caa058ab479540848d2d9793fdfe4e09) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:51.734209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ee77d-7e51-4376-9533-ee5d28c78ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:51.736461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ee77d-7e51-4376-9533-ee5d28c78ca3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.745723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 275/537] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:51 2022] GET /identity/v3/role_assignments?user.id=caa058ab479540848d2d9793fdfe4e09&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:51.746492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc19db4-af36-4080-9608-d608c5417d9d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:51.746991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424a9c3-6168-47fb-8c21-e7f50bf9e431 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.747273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424a9c3-6168-47fb-8c21-e7f50bf9e431 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.747565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424a9c3-6168-47fb-8c21-e7f50bf9e431 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.748938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 276/538] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:51.755343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9679c-040c-4071-a063-7f38790d26c8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.755626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9679c-040c-4071-a063-7f38790d26c8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.755816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9679c-040c-4071-a063-7f38790d26c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.792197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:51.792811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:51.793084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:51.793352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:51.794045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:51.794342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:51.795730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:51.796586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-34d9679c-040c-4071-a063-7f38790d26c8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:51.798389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-34d9679c-040c-4071-a063-7f38790d26c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:51.811030 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 263/539] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:51.818288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:51.920994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 277/540] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:51.938287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:51.977522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24daeb-d4e8-4951-a6c1-abd37e9093e8 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:52.013918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:52.014444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:52.014723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:52.014996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:52.022539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:52.024108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:52.036623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:52.037528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:52.038544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 264/541] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:52.043495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9679c-040c-4071-a063-7f38790d26c8 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:52.054748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:52.147932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9679c-040c-4071-a063-7f38790d26c8 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:52.182888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:52.183564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:52.183848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:52.184125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:52.184830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:52.185136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:52.186589 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:52.205009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 278/542] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:52.211374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:52.299291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:52.319287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfc347-edd3-42e7-a31c-c13c1df5f355 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:52.348638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:52.375466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:52.377144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:52.377824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:52.379394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:52.381005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=b1c427a987ba48878873734ecf0f86e2) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:52.382068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:52.383730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:52.393182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:52.394907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 265/543] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:52 2022] GET /identity/v3/role_assignments?user.id=b1c427a987ba48878873734ecf0f86e2&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:52.455986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a4117-6afe-47a8-bbc8-df854eee6282 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:52.516319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:52.516972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:52.517253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:52.517526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:52.529430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:52.535661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:52.537973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:52.538781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 279/544] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:52.545813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:52.676815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22963fbc-adfa-45ac-98ac-7b4e1856456b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:52.725266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31156d16-843d-4e3d-9e84-464867a3a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:52.726305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31156d16-843d-4e3d-9e84-464867a3a8fc None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:52.726779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31156d16-843d-4e3d-9e84-464867a3a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:52.727240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31156d16-843d-4e3d-9e84-464867a3a8fc None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:52.728097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31156d16-843d-4e3d-9e84-464867a3a8fc None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:52.728583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31156d16-843d-4e3d-9e84-464867a3a8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:52.730181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31156d16-843d-4e3d-9e84-464867a3a8fc None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:52.739759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 266/545] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:52.745737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:52.755064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c13a-88a1-4c49-9611-b7c386fb1de7 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:52.755637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c13a-88a1-4c49-9611-b7c386fb1de7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:52.756103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c13a-88a1-4c49-9611-b7c386fb1de7 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:52.757917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 267/546] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:41:52.763646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf75db-eda8-40b1-9f1d-0af24c8bf372 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:52.764179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf75db-eda8-40b1-9f1d-0af24c8bf372 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:52.764661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf75db-eda8-40b1-9f1d-0af24c8bf372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:52.794423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-1adf75db-eda8-40b1-9f1d-0af24c8bf372 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:52.796928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1adf75db-eda8-40b1-9f1d-0af24c8bf372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:52.903368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:52.922285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 268/547] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:41:52.948380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7bde4-db73-49cb-99bf-244467d3f759 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:52.996013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:53.010467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:53.011119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.011400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.011672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.025294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:53.026783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:53.052734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:53.053728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:53.055103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:53.055103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:53.055103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:53.056325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 280/548] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:53.061727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf75db-eda8-40b1-9f1d-0af24c8bf372 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:53.210251 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf75db-eda8-40b1-9f1d-0af24c8bf372 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:53.271050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:53.292254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:53.292909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.293153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.293870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.294081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:53.294380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:53.295758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:53.318581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 269/549] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:53.325876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:53.431803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:53.495636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616ebeb-4b3b-4c6d-860b-31b4bdd28191 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:53.524386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6904604-68a6-4807-b14b-0f7999ee4349 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:53.561405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:53.562060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.562335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.562602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] PATH_INFO: `/v3/users/swift` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.571434 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:53.572943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:53.585355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:53.586320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:53.587222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 270/550] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:53.593982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:53.607844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:53.608832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.609304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.609845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.638692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:53.644870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:53.645789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:53.646598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:53.655816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 281/551] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:53.672728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:53.718855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fa20e-8757-4a7b-ae40-c8aabd0b5caa None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:53.721522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:53.758448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86353cbb-d417-4fa9-ba0f-fda647e01887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:53.759010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353cbb-d417-4fa9-ba0f-fda647e01887 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.759322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353cbb-d417-4fa9-ba0f-fda647e01887 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.759605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353cbb-d417-4fa9-ba0f-fda647e01887 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.760317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86353cbb-d417-4fa9-ba0f-fda647e01887 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:53.760620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86353cbb-d417-4fa9-ba0f-fda647e01887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:53.765792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86353cbb-d417-4fa9-ba0f-fda647e01887 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:53.768885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:53.788090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 271/552] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Sep 1 13:41:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:53.789915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a24e-2a14-415e-be35-7eb3900de188 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.790204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a24e-2a14-415e-be35-7eb3900de188 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.790475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a24e-2a14-415e-be35-7eb3900de188 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.792350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 272/553] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:53.797006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85a24e-2a14-415e-be35-7eb3900de188 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:53.843559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b8dc2-d0f0-4320-aac4-a2b9cbe06735 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:53.880857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1e557-be90-46f8-bcb5-70fdfc1899f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:53.881471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1e557-be90-46f8-bcb5-70fdfc1899f0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.881762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1e557-be90-46f8-bcb5-70fdfc1899f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.882057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1e557-be90-46f8-bcb5-70fdfc1899f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.882728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1e557-be90-46f8-bcb5-70fdfc1899f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:53.883033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1e557-be90-46f8-bcb5-70fdfc1899f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:53.884427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1e557-be90-46f8-bcb5-70fdfc1899f0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:53.895274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 282/554] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:53.896499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0171-4cec-488d-b279-c3b31d902617 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.896781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0171-4cec-488d-b279-c3b31d902617 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.897053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0171-4cec-488d-b279-c3b31d902617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:53.924671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2a8c0171-4cec-488d-b279-c3b31d902617 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:53.926648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2a8c0171-4cec-488d-b279-c3b31d902617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:53.933019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85a24e-2a14-415e-be35-7eb3900de188 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:53.971658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:53.972415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:53.972819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:53.973195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.009278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.021821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 283/555] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:54.021821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0171-4cec-488d-b279-c3b31d902617 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.052758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.087947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:54.089734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:54.134183 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0171-4cec-488d-b279-c3b31d902617 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:54.172298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:54.173488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.173791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.174052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.186301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:54.187811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:54.206343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:54.207326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:54.208333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:54.208333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:54.208333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:54.208333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 284/556] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:54.223557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.298399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 273/557] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:41:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:41:54.303746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.381205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.394813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.407941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc724f-4595-404c-94b6-877680e2d612 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:54.439093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961994-abcf-462b-a861-12f8bcf0e7e4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:54.450881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c72e2-7d89-47a0-8a72-e8c4cb17f2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:54.451485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c72e2-7d89-47a0-8a72-e8c4cb17f2c0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.451759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c72e2-7d89-47a0-8a72-e8c4cb17f2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.452026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c72e2-7d89-47a0-8a72-e8c4cb17f2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.452691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c72e2-7d89-47a0-8a72-e8c4cb17f2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:54.453060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c72e2-7d89-47a0-8a72-e8c4cb17f2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:54.454996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c72e2-7d89-47a0-8a72-e8c4cb17f2c0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:54.471535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 285/558] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:41:54.471535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545eb956-04a6-4bcc-8b43-a06b77604d99 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.471535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545eb956-04a6-4bcc-8b43-a06b77604d99 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.471535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545eb956-04a6-4bcc-8b43-a06b77604d99 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.471535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 286/559] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:54.471535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545eb956-04a6-4bcc-8b43-a06b77604d99 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.483725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:54.484065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.484396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.484673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.496069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:54.497532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:54.512815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:54.513967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:54.515090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:54.515090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:54.515090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:54.519288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 274/560] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:54.520164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.585188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545eb956-04a6-4bcc-8b43-a06b77604d99 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:54.618253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85891ea0-40e6-4bcc-b62a-d1c0a3815851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:54.618873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85891ea0-40e6-4bcc-b62a-d1c0a3815851 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.619149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85891ea0-40e6-4bcc-b62a-d1c0a3815851 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.619419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85891ea0-40e6-4bcc-b62a-d1c0a3815851 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.623074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831a0b-15c1-472e-b1cc-a8be19af3a3a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:54.630015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85891ea0-40e6-4bcc-b62a-d1c0a3815851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-85891ea0-40e6-4bcc-b62a-d1c0a3815851 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:54.635270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:54.636271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:54.637296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 287/561] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:54.638382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea594a8-43e5-4649-85c5-e5ada4e794fa None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.638382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea594a8-43e5-4649-85c5-e5ada4e794fa None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.638516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea594a8-43e5-4649-85c5-e5ada4e794fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.642880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.646021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.657834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7ea594a8-43e5-4649-85c5-e5ada4e794fa None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:54.659098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7ea594a8-43e5-4649-85c5-e5ada4e794fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:54.663040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:54.663711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.664018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.664292 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.672699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:54.674277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:54.685792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:54.687017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:41:54.688682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:54.689930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 275/562] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:54.690807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.726570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 288/563] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:54.728082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef0b0b-7c04-4b66-941e-996d3f8bd261 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.728443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef0b0b-7c04-4b66-941e-996d3f8bd261 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.728789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef0b0b-7c04-4b66-941e-996d3f8bd261 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.733834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 289/564] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:54.733834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef0b0b-7c04-4b66-941e-996d3f8bd261 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.803151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823c2de-2500-4588-89d6-45a030c0f0d9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:54.838882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:54.839417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.839695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.839964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.840602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:54.840903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:54.842307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:54.852641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef0b0b-7c04-4b66-941e-996d3f8bd261 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:54.859219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 276/565] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:54.859219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:54.892910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.893987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:54.898849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688d877-2665-4f60-a5bc-261c8a2d59df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:54.899527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688d877-2665-4f60-a5bc-261c8a2d59df None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.899923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688d877-2665-4f60-a5bc-261c8a2d59df None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.902455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688d877-2665-4f60-a5bc-261c8a2d59df None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.902455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688d877-2665-4f60-a5bc-261c8a2d59df None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:54.902455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688d877-2665-4f60-a5bc-261c8a2d59df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:54.902692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688d877-2665-4f60-a5bc-261c8a2d59df None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:54.911535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 290/566] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:54.912939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750df667-c926-4fbb-8852-12da6964b54a None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:54.913224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750df667-c926-4fbb-8852-12da6964b54a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:54.913490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750df667-c926-4fbb-8852-12da6964b54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:54.934244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-750df667-c926-4fbb-8852-12da6964b54a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:54.936085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-750df667-c926-4fbb-8852-12da6964b54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:54.970044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceef26-8c03-4715-8220-c48f836ac2c5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.007628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.008246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.008524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.009812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.009956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 291/567] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:55.010050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:55.010050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.010239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750df667-c926-4fbb-8852-12da6964b54a None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.011326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.034757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 277/568] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:41:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:55.034757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.136689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750df667-c926-4fbb-8852-12da6964b54a None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.163161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.163326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.172618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6ff1-08a5-40d0-9a14-c2a861574502 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.181919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.181919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.181919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.181919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.182690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.184358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:55.196754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:55.197724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:55.198779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 292/569] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:55.199257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.211813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.216121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.216121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.216121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.223140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.224588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:55.239422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:55.240365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:55.241238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:55.241238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:55.241238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:55.241238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 278/570] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:55.241475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.315557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145006-7d19-4fbd-9c16-0d1fbb45a7be None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.350836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.351457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.351734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.352002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.355720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc714a-52d7-4960-ad54-1c39450db865 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.362608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:55.367638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:55.368584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:55.369456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 293/571] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:55.370393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.395017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.395585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.395866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.396139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.408385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.409868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.412353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.416732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:55.425239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:55.428992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:55.429934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:55.429934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:55.429934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:55.429934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 279/572] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:55.429934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.472271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9813c31-22d4-42e5-a885-ce604759f168 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.506810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.507421 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.507694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.507962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.508653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:55.508955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.510406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.521660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 294/573] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:55.522486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.528867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ea23-5128-44d2-b8cf-cd51d6f8a63c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.565423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.566059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.566336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.566606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.567246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:55.567543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.569034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.581426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 280/574] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:55.582259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.626988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43d3ca-310c-4536-a54f-e461051fcff6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.629318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.644905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.662574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.663606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.664116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.664685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.676640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:55.681540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:55.682583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 295/575] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:55.683636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.690707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77a4fd-41cd-4cda-9484-2aa74220f846 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.725078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.725691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.726005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.726280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.726947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:55.727251 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.728651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.736487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 281/576] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:55.737298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.786549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971bcaf-730f-4fb2-accf-79fe4669187c None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.820309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.820911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.821185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.821451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.822116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:55.822412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.823758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.835793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 296/577] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:55.836575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.837500 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504c0e-22de-4ad1-8459-1d0a855a6534 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.857199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.863002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:55.873704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.874338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.874615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.874887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.875776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=0f29e56690b343269681dfd8caffe001) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:55.876105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:55.877588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:55.883409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 282/578] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/role_assignments?user.id=0f29e56690b343269681dfd8caffe001&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:55.884056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:55.951097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7b364-6d32-405a-b788-475fc821a2a9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:55.990094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:55.990743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:55.992967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:55.992967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] PATH_INFO: `/v3/users/glance` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:55.999559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.000328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d66d8-bddd-4c9f-b98c-6048801adad6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.001554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:56.017137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:56.018353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 297/579] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:56.019312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.045128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.045742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.046072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.046332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.047000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:56.047324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.048688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.057049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 283/580] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:56.058240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.113248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.119153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.135680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b3f5c-b6fd-40b1-aff2-a3f5be10e29c None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.170845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427032f-92c6-4a51-89ab-ccb38b3af2f6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.171184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.171844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.172266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.173069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.184686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.186217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:56.201586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:56.202806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:56.203920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:56.203920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:56.203920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:56.203920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 298/581] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:56.203920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.215204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.215699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.215978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.216250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.224294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.225790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:56.238689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:56.239786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 284/582] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:56.241309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.324896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf3dec-14eb-4202-819e-fa6967ca5922 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.362207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1ac1-56eb-4ef3-9dba-93127551dd62 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.363917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.364542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.364807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.365078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.365840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:56.366148 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.367431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.367815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.371294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.386679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 299/583] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:56.392519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.402442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.403088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.403372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.403636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.415555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.417688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:56.434983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:56.436236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:56.437440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:56.437440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:56.437440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:56.437440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 285/584] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:56.437440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.503891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e624cc-fcfb-4ffd-8db5-d77b755b4af4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.541459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.542098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.542384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.542818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.543294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:56.543591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.544945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.549278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dcc56-154a-4588-b086-0b1c8b91e85e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.560399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 300/585] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:56.560399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.591920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.592525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.592800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.593070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.593758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:56.594107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.596107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.608332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 286/586] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:56.609062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.623025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.627073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.669880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab575-17c6-4118-8d62-328703bd6e9c None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.706429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.707062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.707339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.707611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.719571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.719737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d45295-e04d-465f-a115-c07461ccf2ef None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.721026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:56.737150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:56.738205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:56.739157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:56.739157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:56.739157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:56.739157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 301/587] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:56.739807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.764435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.764975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.765252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.765522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.766810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:56.767110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.768469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.784913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 287/588] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:56.784913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.850000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62fd03-0c35-4b62-a28d-f6550e0b9165 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.859303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.867970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:56.889244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.890224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.890507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.890784 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.893094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903b6f-db28-431d-9779-241aad9b9c49 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:56.899316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.900779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:56.914304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:56.922100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 302/589] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:56.923159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:56.937121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:56.937810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:56.938752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:56.939121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/19a91e958d514c0d8a61289bbe3ea63d/roles/61fd046fc53c4a57a1ddd3e4f878622c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:56.958762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=19a91e958d514c0d8a61289bbe3ea63d, role_id=61fd046fc53c4a57a1ddd3e4f878622c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:56.960344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.048978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88576ec7-611c-4820-908f-8be4081aab55 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.053402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 288/590] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:56 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/19a91e958d514c0d8a61289bbe3ea63d/roles/61fd046fc53c4a57a1ddd3e4f878622c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:57.054476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.117881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.118710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.118841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.119048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.119696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:57.120014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.121378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.146699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 303/591] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:57.148555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.183835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:57.224295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbff29-13de-4686-8ce3-5580e17a91de None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.269915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:57.287522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.288261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.288545 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.288820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.301612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.304283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.304421 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0b16-9e45-4689-aabe-d3e196a14f97 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:57.322208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:57.323407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:57.324416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 289/592] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:57.325703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.406453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.413158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.417805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.417805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.417805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:57.417805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.417805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.445810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43f13-0d39-4fda-98e8-7781f4749c3a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.470423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 304/593] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:57.475323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.495522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.496832 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.497245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.497899 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.510616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.512085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:57.528160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:57.529138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:57.530067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:57.535349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 290/594] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:57.535511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.585382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:57.597823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:57.614701 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1f483-4b80-4c0a-af6c-652f4032b54d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.641598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6a41-4662-4255-92a5-6c6d4d3516dd None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.656599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.657584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.658107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.658641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.659742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:57.660270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.662079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.676801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.677438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.677719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.678027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.683534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 305/595] 10.222.0.61 () {56 vars in 1139 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:57.685142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.690459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.691731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:57.707681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:57.708679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:57.710230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:57.710230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:57.710230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:57.713603 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 291/596] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:57.714496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.802048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9521e-bdd1-405c-9335-a9aa4a55c4a9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.816932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:57.820896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37632d2-b541-47e9-bd0d-8407f1bec170 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:57.848090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.848090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.848090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.848090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.849255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:57.849255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.849255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.859438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 306/597] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:57.860776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.863313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:57.866924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb7113-cba9-4651-8fa7-93d097eaf2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:57.867987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7113-cba9-4651-8fa7-93d097eaf2aa None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.868419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7113-cba9-4651-8fa7-93d097eaf2aa None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.868862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7113-cba9-4651-8fa7-93d097eaf2aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.880409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb7113-cba9-4651-8fa7-93d097eaf2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:57.881916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb7113-cba9-4651-8fa7-93d097eaf2aa None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-2cbb7113-cba9-4651-8fa7-93d097eaf2aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:57.896320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:57.897515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:57.898449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:57.898449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:57.898449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:57.898449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 292/598] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:57.898870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71335e1b-ab2b-47cb-b0af-d56778361c59 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:57.899143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71335e1b-ab2b-47cb-b0af-d56778361c59 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:57.899408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71335e1b-ab2b-47cb-b0af-d56778361c59 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:57.900743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 293/599] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:57.905196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71335e1b-ab2b-47cb-b0af-d56778361c59 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:57.979688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f483f-db5e-4518-a38d-8501863e2f01 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.012822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71335e1b-ab2b-47cb-b0af-d56778361c59 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.017827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.018694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.019223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.019828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.020562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:58.020936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.022513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.039950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 307/600] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:58.039950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.053066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e5bdf-fe12-4198-95ff-5390dab729cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.053726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e5bdf-fe12-4198-95ff-5390dab729cd None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.054043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e5bdf-fe12-4198-95ff-5390dab729cd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.054314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e5bdf-fe12-4198-95ff-5390dab729cd None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/42f33454f4b94b7fb54e83252ce564db/roles/61fd046fc53c4a57a1ddd3e4f878622c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.070856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e5bdf-fe12-4198-95ff-5390dab729cd None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=42f33454f4b94b7fb54e83252ce564db, role_id=61fd046fc53c4a57a1ddd3e4f878622c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.072283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e5bdf-fe12-4198-95ff-5390dab729cd None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.104068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:58.141974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc36e-2c52-4e36-8919-6b5a6a70159f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.153958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:58.158979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 294/601] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:57 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/42f33454f4b94b7fb54e83252ce564db/roles/61fd046fc53c4a57a1ddd3e4f878622c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:58.160369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ccb6b-ec3b-481a-9a7c-c2d4c8fadc19 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.160789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ccb6b-ec3b-481a-9a7c-c2d4c8fadc19 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.161209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ccb6b-ec3b-481a-9a7c-c2d4c8fadc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.182579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b19ccb6b-ec3b-481a-9a7c-c2d4c8fadc19 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:58.184227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b19ccb6b-ec3b-481a-9a7c-c2d4c8fadc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:58.191885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.192556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.192839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.193107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.193763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:58.194095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.196301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.217933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 308/602] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:58.217933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.280265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 295/603] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:58.282194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ccb6b-ec3b-481a-9a7c-c2d4c8fadc19 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.359830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92862682-6091-4e8e-a720-2c746b9f0cc9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.387410 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:58.400382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.401080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.401371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.401638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.410501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.411598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ccb6b-ec3b-481a-9a7c-c2d4c8fadc19 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.411967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:58.425674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:58.426653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:58.427524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 309/604] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:58.432841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.442764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:58.452181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.452888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.453112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.453803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.467743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.474017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:41:58.491617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:58.494063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:58.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 296/605] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:58.501810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.637671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5ec84-81cc-400b-a978-a761be98a39d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.648552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b67f7b-d00b-436d-8209-c87840d73547 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.676750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.677168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.677450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.677715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.678442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:58.678778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.680147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.686280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.687230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.687759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.688249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.689270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:58.689830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.691479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.692970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 310/606] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:58.693918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.706992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 297/607] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:58.708254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.742813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:58.761482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:58.800053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a80b-0cb3-41d3-87d6-8c45f67d9b1d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.815190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142b03f-048b-443e-bc3c-5cebdb5d482d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.838050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.839090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.839543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.840054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.852572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:41:58.852760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:58.853565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:58.853944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:58.854289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/caa058ab479540848d2d9793fdfe4e09/roles/61fd046fc53c4a57a1ddd3e4f878622c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:41:58.859298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:58.860301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:41:58.861235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:41:58.864080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 311/608] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:58.865127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:58.877534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=caa058ab479540848d2d9793fdfe4e09, role_id=61fd046fc53c4a57a1ddd3e4f878622c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:58.879423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:58.973168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbc3e1-0def-4f8a-b3c6-992e316b0193 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:58.974739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:59.014200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 298/609] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:58 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/caa058ab479540848d2d9793fdfe4e09/roles/61fd046fc53c4a57a1ddd3e4f878622c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:59.017380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:59.028844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbc8e0-1fcd-463b-adc4-317ba2ab34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:59.033717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc8e0-1fcd-463b-adc4-317ba2ab34a2 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:59.034096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc8e0-1fcd-463b-adc4-317ba2ab34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:59.034373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc8e0-1fcd-463b-adc4-317ba2ab34a2 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/b1c427a987ba48878873734ecf0f86e2/roles/61fd046fc53c4a57a1ddd3e4f878622c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:59.072318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbc8e0-1fcd-463b-adc4-317ba2ab34a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=b1c427a987ba48878873734ecf0f86e2, role_id=61fd046fc53c4a57a1ddd3e4f878622c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:59.077644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbc8e0-1fcd-463b-adc4-317ba2ab34a2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:59.170807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:59.174564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d74327-6dfd-421f-b3b6-6f5108d37415 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:59.212889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 312/610] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:41:58 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/b1c427a987ba48878873734ecf0f86e2/roles/61fd046fc53c4a57a1ddd3e4f878622c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:41:59.228914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:59.230205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:59.230663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:59.231104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:59.232021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:59.232615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:59.234466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:59.234958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a744dc-90b9-425e-885b-bcfcf644158e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:59.235252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a744dc-90b9-425e-885b-bcfcf644158e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:59.235554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a744dc-90b9-425e-885b-bcfcf644158e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:59.237133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 313/611] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:41:59.242722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba1a6b-a5a6-4c93-bff7-53c9d55cf38d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:59.243002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba1a6b-a5a6-4c93-bff7-53c9d55cf38d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:59.243278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba1a6b-a5a6-4c93-bff7-53c9d55cf38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:59.246703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 299/612] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:59.253682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:59.278851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b1ba1a6b-a5a6-4c93-bff7-53c9d55cf38d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:41:59.280078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b1ba1a6b-a5a6-4c93-bff7-53c9d55cf38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:41:59.410075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec69be-9e7d-4270-a4e9-349202a92881 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:59.443112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 314/613] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:41:59.460533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:59.463356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:59.463884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:59.464419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:59.476871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:59.478724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:41:59.495338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:59.496314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:41:59.497248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:41:59.497248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:41:59.497248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:59.499223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 300/614] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:41:59.505700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba1a6b-a5a6-4c93-bff7-53c9d55cf38d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:59.594767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:59.615112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba1a6b-a5a6-4c93-bff7-53c9d55cf38d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:59.652924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:59.653889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:59.654338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:59.654842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:59.655643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:41:59.656107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:59.658175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:59.671282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 315/615] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:41:59.677540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:59.695514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:59.780695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:59.791847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd2f4-6e8b-4ffd-9fb7-82ead467b119 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:41:59.833887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:41:59.834806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:41:59.835392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:41:59.835903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:41:59.845761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:41:59.847604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:41:59.869332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:41:59.870826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:41:59.871821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 301/616] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:41:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:41:59.878610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:41:59.956444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:41:59.982393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d434994-556f-4f16-8d45-a6ad27d9b864 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.018092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.018703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.019005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.019254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.019984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:00.020283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.021716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.023952 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d367b-de5c-4e2c-a5cf-432df5cdc224 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.041067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 302/617] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Sep 1 13:41:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:00.048315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.096964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.097875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.098361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.098864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.110448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:00.115979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:00.117360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:00.118595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:00.121020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 316/618] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:00.127555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.132453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:00.227402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660abaa0-cb98-4ac6-862d-a647332153a0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.253681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833aac-eae2-418c-8018-d7d3bd20f6dd None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.257722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:00.263473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.264196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.264531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.264889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.276268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.277803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:00.294263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:00.295259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:00.296150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:00.296150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:00.296150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:00.296150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 303/619] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:00.306332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.319850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae1edf-8a1b-4239-999b-6f05e7e8e010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.320762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae1edf-8a1b-4239-999b-6f05e7e8e010 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.321274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae1edf-8a1b-4239-999b-6f05e7e8e010 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.321855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae1edf-8a1b-4239-999b-6f05e7e8e010 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.322830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae1edf-8a1b-4239-999b-6f05e7e8e010 None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:00.323366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae1edf-8a1b-4239-999b-6f05e7e8e010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.324967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae1edf-8a1b-4239-999b-6f05e7e8e010 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.335160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 317/620] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:00.336751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb474ff-419d-4041-9147-0dc7af2b1e74 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.337295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb474ff-419d-4041-9147-0dc7af2b1e74 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.337826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb474ff-419d-4041-9147-0dc7af2b1e74 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.339386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 318/621] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:00.340655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb474ff-419d-4041-9147-0dc7af2b1e74 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.436501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:00.449146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931aabe-d2a5-4440-a78e-a638b3ef3684 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.481716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb474ff-419d-4041-9147-0dc7af2b1e74 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.486712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad893c-7ee3-47ab-8434-dc2802c6ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.487438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad893c-7ee3-47ab-8434-dc2802c6ec27 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.487731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad893c-7ee3-47ab-8434-dc2802c6ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.488004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad893c-7ee3-47ab-8434-dc2802c6ec27 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.488653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad893c-7ee3-47ab-8434-dc2802c6ec27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:00.489003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad893c-7ee3-47ab-8434-dc2802c6ec27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.490455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad893c-7ee3-47ab-8434-dc2802c6ec27 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.508562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 304/622] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:00.510003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0afe0-621b-428a-99e6-675457de67f6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.510297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0afe0-621b-428a-99e6-675457de67f6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.510646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0afe0-621b-428a-99e6-675457de67f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.522350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.523288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.523754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.524227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.530854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-7ca0afe0-621b-428a-99e6-675457de67f6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:00.532114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7ca0afe0-621b-428a-99e6-675457de67f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:00.537226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.538957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:00.554052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:00.554992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:00.555871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:00.555871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:00.555871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:00.562424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 319/623] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:00.564145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.576887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:00.605467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 305/624] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:00.606480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0afe0-621b-428a-99e6-675457de67f6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.673810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299d4d1-729e-428d-b8b0-27a9f47b8a1a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.691170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:00.710868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.711806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.712314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.712781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.715725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0afe0-621b-428a-99e6-675457de67f6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.719987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.721748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:00.735013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:00.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:00.736819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:00.739343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 320/625] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:00.746310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.756660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.757691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.757691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.757914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.758504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:00.758816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.760233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.773367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 306/626] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:00.783020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.838172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:00.888857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4212f9-13f6-4b10-b9e7-713ab5b6828c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:00.926277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:00.926926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:00.927211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:00.927489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:00.935718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:00.937305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:00.949206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:00.950199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:00.951142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 307/627] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:00.956815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:00.989462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5659-4f6e-4d8e-9927-6ef9ac22476a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.033306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:01.035157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.035447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.035751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.036513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:01.036818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:01.038324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:01.052712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 321/628] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:01.058552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:01.076021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:01.083535 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289110a9-5e6a-4c00-802b-b216ddd0d2fc None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.096759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:01.121400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:01.122594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.123548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.124045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.125956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:01.131550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:01.131550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:01.155577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 308/629] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:01.157812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:01.183081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494b33-889d-4b04-815d-b5f21b536700 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.236813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:01.237745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.238222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.238642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.253926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:01.260535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:01.261804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:01.262845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:01.265850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 322/630] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:01.268054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:01.273323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dfe28-2957-42f8-b8e0-5eee6c30bd69 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.317916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:01.318701 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.319075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.319442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/0f29e56690b343269681dfd8caffe001/roles/61fd046fc53c4a57a1ddd3e4f878622c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.337451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=0f29e56690b343269681dfd8caffe001, role_id=61fd046fc53c4a57a1ddd3e4f878622c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:01.339168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:01.394179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:01.400180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d566b-05c5-427f-853c-69fb66f07496 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.417602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 309/631] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:01 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/0f29e56690b343269681dfd8caffe001/roles/61fd046fc53c4a57a1ddd3e4f878622c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:01.418734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:01.425940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:01.466800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83f1a9-03bb-4c21-a856-063ed6e11f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:01.467433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83f1a9-03bb-4c21-a856-063ed6e11f86 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.467716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83f1a9-03bb-4c21-a856-063ed6e11f86 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.467993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83f1a9-03bb-4c21-a856-063ed6e11f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.553926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83f1a9-03bb-4c21-a856-063ed6e11f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:01.555598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83f1a9-03bb-4c21-a856-063ed6e11f86 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:01.564033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b752970-8d63-4327-9bc9-142cffd143a9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.605556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164599-2464-44ea-b315-1d2afc5af96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:01.606627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164599-2464-44ea-b315-1d2afc5af96b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.606910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164599-2464-44ea-b315-1d2afc5af96b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.607184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164599-2464-44ea-b315-1d2afc5af96b None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.607865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164599-2464-44ea-b315-1d2afc5af96b None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:01.608171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164599-2464-44ea-b315-1d2afc5af96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:01.609573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164599-2464-44ea-b315-1d2afc5af96b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:01.618360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 310/632] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:01.619463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de4ee9-f26d-4cfd-aac7-c34bf467287e None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.619750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de4ee9-f26d-4cfd-aac7-c34bf467287e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.620024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de4ee9-f26d-4cfd-aac7-c34bf467287e None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.621367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 311/633] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:01.622261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314bf6c-93be-400f-a917-bdc3e88c83b5 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.622542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314bf6c-93be-400f-a917-bdc3e88c83b5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.622818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314bf6c-93be-400f-a917-bdc3e88c83b5 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.624028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 312/634] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:01.624732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314bf6c-93be-400f-a917-bdc3e88c83b5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:01.669909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:01.681437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 323/635] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:01.681437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d740-48e4-40f6-9ee8-cd38d05e9993 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.681437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d740-48e4-40f6-9ee8-cd38d05e9993 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.681437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d740-48e4-40f6-9ee8-cd38d05e9993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.696630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a6e3d740-48e4-40f6-9ee8-cd38d05e9993 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:01.697932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a6e3d740-48e4-40f6-9ee8-cd38d05e9993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:01.700092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:01.740637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314bf6c-93be-400f-a917-bdc3e88c83b5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.776623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:01.777261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.777539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.777836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.782374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 324/636] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:01.784022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d101b5d-aa6e-419c-9c7b-1d570cc29af8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:01.784525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d101b5d-aa6e-419c-9c7b-1d570cc29af8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:01.785039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d101b5d-aa6e-419c-9c7b-1d570cc29af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:01.790028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:01.791548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:01.806954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:01.808091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:01.808988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:01.808988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:01.808988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:01.808988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 313/637] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:01.809884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:01.810807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2d101b5d-aa6e-419c-9c7b-1d570cc29af8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:01.811970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2d101b5d-aa6e-419c-9c7b-1d570cc29af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:01.888848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 325/638] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:01.889914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d101b5d-aa6e-419c-9c7b-1d570cc29af8 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:01.955744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:01.961417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22f0c9-e7bb-4545-8208-b62f1df77f5c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:01.983953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:02.004614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.005544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.005960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.006312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.008719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d101b5d-aa6e-419c-9c7b-1d570cc29af8 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.018562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.020249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:02.035658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:02.036880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:02.038033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:02.038033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:02.038033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:02.042011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 314/639] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:02.045712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.056447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.057104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.057386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.057658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.058366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:02.058749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.060608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.078000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 326/640] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:02.086722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.160511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cedfbc-3d8e-4c23-92a6-8b72790b6c44 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.196662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.197626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.198157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.198638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.199484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:02.200004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.201653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.219505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 315/641] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:02.220696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.225253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f881d71-f1b4-4c46-b2f0-1fe88461533d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.246066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:02.265369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:02.268100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.284686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.284686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.284686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] PATH_INFO: `/v3/users/glance` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.284686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.284686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:02.297819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:02.299026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 13:42:02.300190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:02.302280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 327/642] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:02.303290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.351708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e160e-5d3c-4180-a420-c7f7c1bc8bbe None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.391840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.392482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.392760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.393027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.407926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:02.409649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441173-1346-45e0-9718-f1e886b9162f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:02.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:02.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:02.414611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:02.418636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 316/643] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:02.419659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.448294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.448967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.449246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.449519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.456855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.458327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:02.470510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:02.471428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 328/644] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:02.472491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.516814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:02.523095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:02.535294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7ed16-a305-473f-b1ff-4be63df1b7aa None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.575466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.576136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.576367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.576640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.589901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d756f-21e3-4702-b05d-3e6a8de837ad None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.591660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:02.598098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:02.598989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:02.599831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:02.601092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 317/645] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:02.601995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.630100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.631030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.631515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.631995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.632913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:02.633408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.635087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.653052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 329/646] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:02.654211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.709401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378db28-6245-4cb0-8cf4-4604b7cfda1c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.750440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.750964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.751242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.751513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.752181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:02.752485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.753886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.761987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:02.765982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 318/647] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:02.765982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.772342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb8f69-be1f-4055-99a3-d04050f8514b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.779019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:02.807429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.808036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.808314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.808583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.809278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:02.809580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.811059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.827827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 330/648] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:02.827827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.879420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c5b50-fead-425c-bd71-d75c5eae7aad None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.915478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.916090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.916793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.917070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:02.917741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:02.918076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:02.920145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:02.930311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 319/649] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:02.935526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:02.942366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4095c-6b52-47d9-95f2-3aad63fda264 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:02.980743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:02.981353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:02.981629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:02.988293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.012649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.018314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.042894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a86cf-d7fe-4b90-9b9e-6e27e9d99ff6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.063392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.064827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.079515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.080146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.080435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.080695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.092140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.093679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:03.108762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:03.109754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:03.110702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:03.110702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:03.110702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:03.110702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 320/650] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:03.110702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.179553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 331/651] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:03.180925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.229726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11434d-4ee5-42f3-817f-7e37b0eedfcb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.239832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.254164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.264810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.265349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.266165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.266454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.267183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=19a91e958d514c0d8a61289bbe3ea63d) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:03.267482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.268963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.285443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 321/652] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/role_assignments?user.id=19a91e958d514c0d8a61289bbe3ea63d&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:03.285443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.297914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f201b-5b23-4d14-b72d-553acc764cab None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.348971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.349614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.349921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.350191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.365152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.366767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:03.383506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:03.384565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:03.385466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:03.385466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:03.385466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:03.385466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 332/653] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:03.385466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.428565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9caadf-05be-43c2-9b89-17071846c18f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.473298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d6d9c-7b31-41f0-a8b2-311b0a9c634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.473968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d6d9c-7b31-41f0-a8b2-311b0a9c634a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.474459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d6d9c-7b31-41f0-a8b2-311b0a9c634a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.474745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d6d9c-7b31-41f0-a8b2-311b0a9c634a None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.475478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d6d9c-7b31-41f0-a8b2-311b0a9c634a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:03.476522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d6d9c-7b31-41f0-a8b2-311b0a9c634a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.477980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d6d9c-7b31-41f0-a8b2-311b0a9c634a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.493017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 322/654] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:03.495123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54de74-3007-4fe5-b544-0fcaccd35e33 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.495415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54de74-3007-4fe5-b544-0fcaccd35e33 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.495684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54de74-3007-4fe5-b544-0fcaccd35e33 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.497191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 323/655] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:03.502199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54de74-3007-4fe5-b544-0fcaccd35e33 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.513599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48ed15-d0a9-4199-8a3c-65cd105b1a13 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.522174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.549836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.550469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.550746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.551012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.557654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.563431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.565284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:03.582096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:03.583454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:03.584796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:03.584796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:03.584796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:03.584796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 333/656] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:03.584796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.614318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54de74-3007-4fe5-b544-0fcaccd35e33 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.650996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c4880-6494-4884-ac35-9b4c6755fcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.651671 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c4880-6494-4884-ac35-9b4c6755fcb0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.651958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c4880-6494-4884-ac35-9b4c6755fcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.652235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c4880-6494-4884-ac35-9b4c6755fcb0 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.652883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c4880-6494-4884-ac35-9b4c6755fcb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:03.653186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c4880-6494-4884-ac35-9b4c6755fcb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.654611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c4880-6494-4884-ac35-9b4c6755fcb0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.670501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 324/657] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:03.670501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60106710-be96-4b64-95a5-0e6d339b9681 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.670501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60106710-be96-4b64-95a5-0e6d339b9681 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.670501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60106710-be96-4b64-95a5-0e6d339b9681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.687256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-60106710-be96-4b64-95a5-0e6d339b9681 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:03.689027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-60106710-be96-4b64-95a5-0e6d339b9681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:03.705312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd40bc-0079-4f3a-a8c4-b31b59da696f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.740727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.741359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.741641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.741937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.751029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.752468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.765814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 325/658] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:03.765814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60106710-be96-4b64-95a5-0e6d339b9681 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:03.770265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:03.771194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 334/659] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:03.772239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.795974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.817062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:03.891833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f022f-09eb-4e01-a51b-74b7bbd17ec9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.902148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60106710-be96-4b64-95a5-0e6d339b9681 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:03.940324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.940532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:03.941190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.941487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:03.941599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.942029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.942140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:03.942549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:03.942909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:03.943333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.944850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.951372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:03.953687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:03.962093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 326/660] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:03.963161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:03.968744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:03.969890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 13:42:03.971055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:03.973278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 335/661] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:03.974227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.080161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.083245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.084201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241c0f-9744-4c6d-ba46-b765ef907e2e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.105568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ae7c9-b106-44dd-bac2-45d1f5e868ac None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.119220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.119840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.120118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.120390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.121126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:04.121718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.123429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.141515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.142169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 327/662] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:04.143066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.146565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.146852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.147127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.153989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.155404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:04.167705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:04.168756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 336/663] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:04.169630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.252727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc78a-c79c-4bc7-bd31-b6cd5f90be6e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.284320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22805280-d816-42b9-a6dc-c5289595ddc8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.292835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.293460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.293738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.294075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.294818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:04.295127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.296654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.317237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 328/664] 10.222.0.61 () {56 vars in 1139 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:04.318592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.325745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.326401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.326681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.326951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.335798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.339779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.340258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:04.345750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:04.347306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 337/665] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:04.348141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.434887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59858c-f870-4e63-8f9e-8a6d72e2d40f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.455790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d65004-9dd8-4e23-8c80-7c9f0f2c7527 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.473908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.474584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.474904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.475179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.494503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.495107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.495383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.495652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.496336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:04.496641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.498087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.511782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 338/666] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:04.514652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.550482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.551957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.567784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.581935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.623118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4f7c7-35a6-4281-8887-1b04ce2e0f47 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.663000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 329/667] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:04.663000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.664828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.665568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.665876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.666157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.744267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.745919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.762263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642592c1-9371-4c26-9176-d72433909c77 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.776515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.797159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.808688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.808688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.808688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.808688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:04.808688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.808688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.812434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:04.813530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 330/668] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:04.814320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.864922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 339/669] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:04.866208 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.922054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71288f7a-09ec-494f-8b37-5119f8b71499 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:04.958877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:04.959491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:04.959773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:04.960048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:04.960924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=42f33454f4b94b7fb54e83252ce564db) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:04.961236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:04.962718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:04.977804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 331/670] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/role_assignments?user.id=42f33454f4b94b7fb54e83252ce564db&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:04.977804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:04.982976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b72959-941d-4a60-b14b-9f3fcb839159 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.030787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.034754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.035600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.035890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.036164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.051888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.053465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:05.071317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:05.072339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:05.073264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:05.073264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:05.073264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:05.077253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 340/671] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:05.079084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.093251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.125758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660671cd-4a7a-4551-8136-01ce7c2f57a2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.174043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.174598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.174876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.175151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.188123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.189426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.205762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6d203-e063-4116-9efc-c97b455aa4f7 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:05.207678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:05.208646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:05.209632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:05.209632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:05.209632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:05.209900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 332/672] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:05.211377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.254548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.255191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.255473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.255802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.268668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.270340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:05.293414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:05.294458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:05.295364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:05.295364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:05.295364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:05.295364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 341/673] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:05.300413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.329503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.338518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe46b2-3511-4814-8caf-c9b3ce74c5f0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.373583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.375300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27af0f-5844-4b8b-a18f-e4c8174ba298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.375922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27af0f-5844-4b8b-a18f-e4c8174ba298 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.376202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27af0f-5844-4b8b-a18f-e4c8174ba298 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.376465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27af0f-5844-4b8b-a18f-e4c8174ba298 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.377115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27af0f-5844-4b8b-a18f-e4c8174ba298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:05.377415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27af0f-5844-4b8b-a18f-e4c8174ba298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.378827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27af0f-5844-4b8b-a18f-e4c8174ba298 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.395745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 333/674] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:05.396889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d7648-4983-4d2b-8005-6c72d4c04e5e None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.397181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d7648-4983-4d2b-8005-6c72d4c04e5e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.397490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d7648-4983-4d2b-8005-6c72d4c04e5e None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.399080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 334/675] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:05.399886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d7648-4983-4d2b-8005-6c72d4c04e5e None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.421600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1d10-fce8-483b-ae86-4fcc5d73d869 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.458199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.458889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.459266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.459658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.460408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:05.460810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.462310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.475602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 342/676] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:05.476317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.592509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b71a3-ec08-4373-9e0a-123511a7bb98 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.618604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.628339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d7648-4983-4d2b-8005-6c72d4c04e5e None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.633184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5b8e2-5dc7-4ca3-b070-d800d4398f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.633921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5b8e2-5dc7-4ca3-b070-d800d4398f4d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.634275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5b8e2-5dc7-4ca3-b070-d800d4398f4d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.634618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5b8e2-5dc7-4ca3-b070-d800d4398f4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.644315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5b8e2-5dc7-4ca3-b070-d800d4398f4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.646472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5b8e2-5dc7-4ca3-b070-d800d4398f4d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-83a5b8e2-5dc7-4ca3-b070-d800d4398f4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:05.660188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:05.661295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:05.662396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:05.663178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 343/677] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:05.667953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.668696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.668838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f03add-4bb6-45d7-b4a8-c3057403ea2d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.669090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f03add-4bb6-45d7-b4a8-c3057403ea2d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.669233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.669384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f03add-4bb6-45d7-b4a8-c3057403ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.669551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.670308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:05.670673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.672133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.687334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b1f03add-4bb6-45d7-b4a8-c3057403ea2d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:05.689237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 335/678] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:05.689367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b1f03add-4bb6-45d7-b4a8-c3057403ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:05.689922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.705695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.759326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 344/679] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:05.760244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f03add-4bb6-45d7-b4a8-c3057403ea2d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.804400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3238a0f-79f5-473c-a37c-9943f0ff1260 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.842217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.842885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.843226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.843497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] PATH_INFO: `/v3/users/swift` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.851618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.853103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.863167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f03add-4bb6-45d7-b4a8-c3057403ea2d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:05.866083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:05.867490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:05.868625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:05.872156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 336/680] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:05.873009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.880799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.902343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db09c86-3b68-4572-97df-05061014404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:05.905409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db09c86-3b68-4572-97df-05061014404d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:05.905409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db09c86-3b68-4572-97df-05061014404d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:05.905409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db09c86-3b68-4572-97df-05061014404d None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:05.905409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db09c86-3b68-4572-97df-05061014404d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:05.905409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db09c86-3b68-4572-97df-05061014404d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:05.907055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db09c86-3b68-4572-97df-05061014404d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:05.919017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 345/681] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:05.920027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db09c86-3b68-4572-97df-05061014404d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:05.959036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:05.993004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8fd54-7946-48c5-a2f9-edaf7e516dcf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.023870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db09c86-3b68-4572-97df-05061014404d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.030430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.031798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.032151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.032528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.040550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.042707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:06.055318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:06.056484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:06.057558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:06.058412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 337/682] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:06.059227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.067419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.068952 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.069307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.069651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.070550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:06.070928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.072443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.093040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 346/683] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Sep 1 13:42:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:06.094525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.129061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:06.182673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2086871-4eb0-4154-bfb9-b3238fdeaf74 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.202075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd58477-7b20-4c8d-9dbf-4e0827e46639 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.222552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:06.227787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.228458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.228806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.229074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.229820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=caa058ab479540848d2d9793fdfe4e09) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:06.230124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.231552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.237383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 338/684] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/role_assignments?user.id=caa058ab479540848d2d9793fdfe4e09&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:06.238024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.251603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.253037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.253508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.254160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.271862 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:06.278910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:06.279930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 347/685] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:06.280783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.380272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc309f-aa5f-4f12-8c02-fa29ef75f9d2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.408854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:06.423696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ddac-227f-418d-b814-52c6719dc17f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.428506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.429118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.429392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.429659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.430939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:06.430939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.432242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.448644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 339/686] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:06.450391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.471915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.472691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.473275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.473556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.531740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:06.552573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.553505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.588968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38cab3-dba3-4210-8a64-38342a194731 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.626070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.626684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.626962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.627238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.627905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:06.628206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.629587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.641994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 340/687] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:06.641994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.698771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 348/688] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:06.700448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.702080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:06.753678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02018-4dd8-47cf-864a-4324b451bcef None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.776217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:06.790613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.791253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.791602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.791942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.792836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=b1c427a987ba48878873734ecf0f86e2) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:06.793302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.794853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.801905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 341/689] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/role_assignments?user.id=b1c427a987ba48878873734ecf0f86e2&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:06.803033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.809606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d52656-043f-4772-b7a5-ecb25754ca99 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.848876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.849566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.849895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.850176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.862185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.863757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:06.880674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:06.881702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:06.883056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:06.883056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:06.883056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:06.883056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 349/690] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:06.888178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:06.915154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0022984-3a9b-49c6-b8e4-654247a1e98a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:06.935934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:06.955033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:06.955967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:06.956393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:06.956813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:06.969015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:06.970837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:06.989823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:06.990882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:06.991864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:06.991864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:06.991864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:06.994089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 342/691] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:07.001332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.034635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026453f7-65bd-40b5-a4f8-f4319d4a8ca6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.042874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:07.076521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.077465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.077972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.078398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.079259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:07.079810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.081597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.096316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 350/692] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:07.101553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.116868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd7d5e-e2ed-406e-95f6-87568f286a63 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.155751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.157266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.157648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.158042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.158789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:07.159212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.160763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.173633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 343/693] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:07.182707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.223240 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9089f-85cc-44b8-b9c3-61b14711e609 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.225124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:07.268668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaa5f4-8083-4f4e-b16f-631998c82f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.269646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaa5f4-8083-4f4e-b16f-631998c82f0f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.270198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaa5f4-8083-4f4e-b16f-631998c82f0f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.270692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaa5f4-8083-4f4e-b16f-631998c82f0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.283115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaa5f4-8083-4f4e-b16f-631998c82f0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.284839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaa5f4-8083-4f4e-b16f-631998c82f0f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.289212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-eefaa5f4-8083-4f4e-b16f-631998c82f0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:07.302425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:07.303562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:07.304546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:07.305105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c13cfd-0f8b-40ed-b641-5cee764f66c5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.305178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 351/694] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:42:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:07.305178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8be2d4-e069-46d7-b037-3fe80a6718dd None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.305178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8be2d4-e069-46d7-b037-3fe80a6718dd None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.305178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8be2d4-e069-46d7-b037-3fe80a6718dd None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.305178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 352/695] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:07.315415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0dcc-6b6c-4943-b5cd-1255cab86b9b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.315546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0dcc-6b6c-4943-b5cd-1255cab86b9b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.315762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0dcc-6b6c-4943-b5cd-1255cab86b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.336437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-72cf0dcc-6b6c-4943-b5cd-1255cab86b9b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:07.337751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-72cf0dcc-6b6c-4943-b5cd-1255cab86b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:07.345741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.347029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.347408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.347780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.357054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.359309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:07.377939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:07.379041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:07.380039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:07.382785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 344/696] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:07.383695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.418263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 353/697] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:07.426894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf0dcc-6b6c-4943-b5cd-1255cab86b9b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.508338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f813d8-ee44-482a-ab84-24b859db18e8 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.547346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.547961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.548237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.548509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.549237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:07.549539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.551402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.572079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 345/698] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:42:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:07.577342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.594210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:07.622238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:07.646607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf0dcc-6b6c-4943-b5cd-1255cab86b9b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.711795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.712418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.712695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.713044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.713804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:07.714118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.716305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.727242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c3472-f5f4-4302-a811-c09cd47fc86c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.740413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 354/699] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:07.747101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.792624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.793642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.793968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.794505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.868846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09df2e7-ef9f-4cb7-b28b-bb9c9f4de6ad None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:07.889752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.891914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.909752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:07.911305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:07.912151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:07.912645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:07.913128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:07.914097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=0f29e56690b343269681dfd8caffe001) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:07.914603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:07.916225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:07.923792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 355/700] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:07 2022] GET /identity/v3/role_assignments?user.id=0f29e56690b343269681dfd8caffe001&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:07.924932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:07.947690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:08.022579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 346/701] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:08.028019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:08.148970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8322c6-a404-4d93-b5b5-1b3cb4253465 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.149281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07615e-9d0e-4ae5-b79d-d59d7e00f726 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.166218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:08.188565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.189200 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.189530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.189747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.193443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.194107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.194383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.194653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.201342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.202928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:08.208677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.210398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:08.225490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:08.226610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:08.227895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:08.227895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:08.227895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:08.227895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 356/702] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:08.233216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:08.239733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:08.240722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:08.241659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:08.241659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:08.241659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:08.241659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 347/703] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:08.247155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:08.315678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:08.356980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf12a3-ce76-44f6-88a6-d8e507911d93 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.358171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeff6dd-b709-4bec-9ec2-9c10266c1942 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.397167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.397563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.397814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.398265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.398653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.398922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.399411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.399567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:08.399965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.400115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.400906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:08.401366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.401754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:08.403082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:08.415981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 348/704] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:08.417806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 357/705] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:08.423250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:08.424125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:08.458221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:08.528809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78983b-8f60-42ae-bd2f-538828ad1314 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.532226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4216989-5b83-4a82-abae-39c64e13804b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.560204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:08.564760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcb608-0b2b-49df-8f56-a203d0015b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.565530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcb608-0b2b-49df-8f56-a203d0015b43 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.565940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcb608-0b2b-49df-8f56-a203d0015b43 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.566342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcb608-0b2b-49df-8f56-a203d0015b43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.570056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de64b35-4725-45c0-aa0b-0bac6e787eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.570838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de64b35-4725-45c0-aa0b-0bac6e787eda None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.571268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de64b35-4725-45c0-aa0b-0bac6e787eda None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.571719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de64b35-4725-45c0-aa0b-0bac6e787eda None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.572529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de64b35-4725-45c0-aa0b-0bac6e787eda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.574016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcb608-0b2b-49df-8f56-a203d0015b43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.574293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de64b35-4725-45c0-aa0b-0bac6e787eda None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:08.577816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcb608-0b2b-49df-8f56-a203d0015b43 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-18dcb608-0b2b-49df-8f56-a203d0015b43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:08.588192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:08.589303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:08.590426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:08.592025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 349/706] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:08.598306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560de810-e4f2-44ba-99e6-b47bbaa55d56 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.606218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560de810-e4f2-44ba-99e6-b47bbaa55d56 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.606218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560de810-e4f2-44ba-99e6-b47bbaa55d56 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.606218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 350/707] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:08.606218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560de810-e4f2-44ba-99e6-b47bbaa55d56 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:08.665949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 358/708] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:42:08 2022] POST /identity/v3/users => generated 254 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:08.667403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3c6a-6d91-41f6-8ea0-2ed07ebabdd2 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.667827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3c6a-6d91-41f6-8ea0-2ed07ebabdd2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.668267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3c6a-6d91-41f6-8ea0-2ed07ebabdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.700473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c87a3c6a-6d91-41f6-8ea0-2ed07ebabdd2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:08.703714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c87a3c6a-6d91-41f6-8ea0-2ed07ebabdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:08.733495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:08.755967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560de810-e4f2-44ba-99e6-b47bbaa55d56 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.809781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.810515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.810842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.811182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.812089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:08.812987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.814651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:08.836217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 351/709] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:08.841597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:08.846616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 359/710] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:08.954439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df13210-b009-4c22-832f-940f2503a53e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:08.992607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf819957-1e63-4afd-add9-8564aefbffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:08.993609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819957-1e63-4afd-add9-8564aefbffaa None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:08.994130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819957-1e63-4afd-add9-8564aefbffaa None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:08.994602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819957-1e63-4afd-add9-8564aefbffaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:08.995529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf819957-1e63-4afd-add9-8564aefbffaa None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=19a91e958d514c0d8a61289bbe3ea63d) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:08.996140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf819957-1e63-4afd-add9-8564aefbffaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:08.997788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf819957-1e63-4afd-add9-8564aefbffaa None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:09.004426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 352/711] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:08 2022] GET /identity/v3/role_assignments?user.id=19a91e958d514c0d8a61289bbe3ea63d&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:09.179101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3c6a-6d91-41f6-8ea0-2ed07ebabdd2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:09.261381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2fd2b-add8-46e7-91be-b7cc7f94a25d None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:09.261682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2fd2b-add8-46e7-91be-b7cc7f94a25d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:09.261898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2fd2b-add8-46e7-91be-b7cc7f94a25d None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:09.264560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 353/712] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:09.269521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be52d7-34dc-4fb3-b297-08c42a74d22d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:09.269904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be52d7-34dc-4fb3-b297-08c42a74d22d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:09.270352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be52d7-34dc-4fb3-b297-08c42a74d22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:09.295215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3c6a-6d91-41f6-8ea0-2ed07ebabdd2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:09.304873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-07be52d7-34dc-4fb3-b297-08c42a74d22d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:09.304873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-07be52d7-34dc-4fb3-b297-08c42a74d22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:09.307130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:09.330150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:09.365791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:09.366436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:09.366717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:09.367054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:09.367907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:09.369268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:09.391881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:09.392957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:09.394514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 360/713] 10.222.0.61 () {56 vars in 1129 bytes} [Thu Sep 1 13:42:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:09.405417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:09.460659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 354/714] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:09.549833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700f3f4-d4bb-4a33-9fde-8b4d6b1b1a84 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:09.599251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:09.599882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:09.600156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:09.600424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:09.601057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:09.601354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:09.602739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:09.613874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 361/715] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Sep 1 13:42:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:09.619448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be52d7-34dc-4fb3-b297-08c42a74d22d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:09.768976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be52d7-34dc-4fb3-b297-08c42a74d22d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:09.809870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:09.810474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:09.810706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:09.810928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:09.811558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:09.811858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:09.813795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:09.819789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 355/716] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Sep 1 13:42:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:09.842754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:09.903509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:09.979666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c31ed7-290e-4821-90f2-1c3483ca34ed None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:10.026981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:10.027972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:10.028472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:10.028954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] PATH_INFO: `/v3/services/network` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:10.029739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:10.031341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:10.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:10.043662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:10.044494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:10.046340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 362/717] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:09 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:10.052295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:10.055640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:10.167965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78561e-fb68-4d91-93fd-cafbf0678208 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:10.210770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cfee5-2f3e-43e0-9938-f2d7e6021ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:10.211383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cfee5-2f3e-43e0-9938-f2d7e6021ced None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:10.211659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cfee5-2f3e-43e0-9938-f2d7e6021ced None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:10.211926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cfee5-2f3e-43e0-9938-f2d7e6021ced None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:10.212554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cfee5-2f3e-43e0-9938-f2d7e6021ced None admin] RBAC: Adding query filter params (name=network) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:10.212854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cfee5-2f3e-43e0-9938-f2d7e6021ced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:10.214248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cfee5-2f3e-43e0-9938-f2d7e6021ced None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:10.219800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 356/718] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:10.227477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:10.350820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68128fd-d723-4836-b814-f34e78df37a6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:10.358052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:10.385763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a2779-c31f-43aa-ac6f-72dd346cda64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:10.386398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a2779-c31f-43aa-ac6f-72dd346cda64 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:10.386677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a2779-c31f-43aa-ac6f-72dd346cda64 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:10.386946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a2779-c31f-43aa-ac6f-72dd346cda64 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:10.387575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a2779-c31f-43aa-ac6f-72dd346cda64 None admin] RBAC: Adding query filter params (type=network) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:10.387875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a2779-c31f-43aa-ac6f-72dd346cda64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:10.389404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a2779-c31f-43aa-ac6f-72dd346cda64 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:10.395312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 363/719] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:10.671644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe7da-d50b-4a52-b6a3-849f34d8af54 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:10.672038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe7da-d50b-4a52-b6a3-849f34d8af54 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:10.672216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe7da-d50b-4a52-b6a3-849f34d8af54 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:10.673805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 357/720] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:10.679120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a11785-a979-4932-b7f6-6ebca7e7f804 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:10.679408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a11785-a979-4932-b7f6-6ebca7e7f804 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:10.679682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a11785-a979-4932-b7f6-6ebca7e7f804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:10.706101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d4a11785-a979-4932-b7f6-6ebca7e7f804 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:10.707310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d4a11785-a979-4932-b7f6-6ebca7e7f804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:10.793385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 364/721] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:11.190000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c17f8-e37d-4be2-8433-38289b6e77d4 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.190522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c17f8-e37d-4be2-8433-38289b6e77d4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.190522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c17f8-e37d-4be2-8433-38289b6e77d4 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.192116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 358/722] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:11.197399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e990a3-a3f7-4085-92cd-d28bfc61e414 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.197681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e990a3-a3f7-4085-92cd-d28bfc61e414 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.197980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e990a3-a3f7-4085-92cd-d28bfc61e414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.200579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c17f8-e37d-4be2-8433-38289b6e77d4 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:11.232052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-23e990a3-a3f7-4085-92cd-d28bfc61e414 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:11.233228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-23e990a3-a3f7-4085-92cd-d28bfc61e414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:11.321300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:11.359701 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 365/723] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 13:42:11.381122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c17f8-e37d-4be2-8433-38289b6e77d4 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:11.433241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb381b-db59-4eeb-9987-2debf165d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:11.433903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb381b-db59-4eeb-9987-2debf165d96d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.434177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb381b-db59-4eeb-9987-2debf165d96d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.434444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb381b-db59-4eeb-9987-2debf165d96d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.447579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb381b-db59-4eeb-9987-2debf165d96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b3cb381b-db59-4eeb-9987-2debf165d96d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:11.453403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:11.454433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:11.455741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 359/724] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:11.460822 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e990a3-a3f7-4085-92cd-d28bfc61e414 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:11.491590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:11.498050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dbc84-8c7d-4588-8a9d-227ee8b7fa2f None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.498326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dbc84-8c7d-4588-8a9d-227ee8b7fa2f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.498594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dbc84-8c7d-4588-8a9d-227ee8b7fa2f None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.500114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 360/725] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:11.504814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e425c-d645-4471-803a-d0e42c8e8253 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.504939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e425c-d645-4471-803a-d0e42c8e8253 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.505187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e425c-d645-4471-803a-d0e42c8e8253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.529383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f96e425c-d645-4471-803a-d0e42c8e8253 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:11.530578 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f96e425c-d645-4471-803a-d0e42c8e8253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:11.616800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e990a3-a3f7-4085-92cd-d28bfc61e414 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:11.669443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 361/726] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:11.670300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e425c-d645-4471-803a-d0e42c8e8253 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:11.679611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:11.680227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.680504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.680848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.681525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:11.681868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:11.683398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:11.704600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 366/727] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:11.711223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:11.733866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:11.783703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e425c-d645-4471-803a-d0e42c8e8253 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:11.819482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:11.820454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.820950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.821443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.833212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:11.838934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:11.839905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:11.840800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:11.842865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 362/728] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:11.848856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:11.896649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:11.911964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1baf9f-079c-4407-94c5-e6734c7f68b3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:11.983588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:11.984233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:11.984515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:11.984788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:11.998525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.001070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.006306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05992f2d-ebbd-4e05-a479-036e61e3c1bc None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:12.020921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:12.025905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:12.026812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:12.026812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:12.026812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:12.026812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 367/729] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:12.027048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:12.027795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.047235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.052044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.052044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.052044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.052044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:12.052044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.052044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.059520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 363/730] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:12.061877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.178792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12097-ecd2-4f23-ba71-dfea2426ea03 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.194366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e23e8-f724-43b8-b91b-356042a12154 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.224508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.225854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.226336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.226819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.227660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:12.228222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.230163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.231748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.232699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.233190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.233281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.250162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:12.250577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 364/731] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:12.251750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:12.255466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:12.256452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:12.257279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:12.261868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 368/732] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:12.262983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.269256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:12.318190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:12.390610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a764a5-7ca4-4819-8488-147491178628 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.394051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02268c-e9f1-4c40-8ade-854d08731dac None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.431672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.431881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659fff5-c6ad-48e2-95cf-be6b13aa79fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.432475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659fff5-c6ad-48e2-95cf-be6b13aa79fb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.432603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.432757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659fff5-c6ad-48e2-95cf-be6b13aa79fb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.432852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.433016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659fff5-c6ad-48e2-95cf-be6b13aa79fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.433110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.440699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.442226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.443739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659fff5-c6ad-48e2-95cf-be6b13aa79fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.445849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659fff5-c6ad-48e2-95cf-be6b13aa79fb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:12.453658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:12.454630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 369/733] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:12.455588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a659fff5-c6ad-48e2-95cf-be6b13aa79fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:12.465604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:12.466803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:12.467690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:12.467690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:12.467690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:12.467690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 365/734] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:12.467918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8940-86cf-4f4a-abc6-5d727776303a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.469119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8940-86cf-4f4a-abc6-5d727776303a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.469119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8940-86cf-4f4a-abc6-5d727776303a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.474504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 366/735] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:12.474504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a8940-86cf-4f4a-abc6-5d727776303a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.532295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:12.574861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:12.575916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae5825-3ae5-433e-b10b-097c8b936c91 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.587430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a8940-86cf-4f4a-abc6-5d727776303a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.612043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.612940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.613218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.613499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.614205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:12.614510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.615896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.624344 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 370/736] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:12.625149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.633459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88395afd-adba-431a-af44-223f662d6fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.634108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88395afd-adba-431a-af44-223f662d6fd9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.634381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88395afd-adba-431a-af44-223f662d6fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.634647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88395afd-adba-431a-af44-223f662d6fd9 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.635382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88395afd-adba-431a-af44-223f662d6fd9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:12.635687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88395afd-adba-431a-af44-223f662d6fd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.637160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88395afd-adba-431a-af44-223f662d6fd9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.655018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 367/737] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:12.656632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec014e-32f2-497e-a8b3-8cd3ba06b1ee None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.656908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec014e-32f2-497e-a8b3-8cd3ba06b1ee None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.657173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec014e-32f2-497e-a8b3-8cd3ba06b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.675121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2fec014e-32f2-497e-a8b3-8cd3ba06b1ee None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:12.676355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2fec014e-32f2-497e-a8b3-8cd3ba06b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:12.729353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19601-801a-4221-b014-e2131a4dd15f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.744027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 368/738] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:12.744868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec014e-32f2-497e-a8b3-8cd3ba06b1ee None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.765357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:12.774302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193840-2141-4a87-9ead-a89c861476e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.774914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193840-2141-4a87-9ead-a89c861476e3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.775225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193840-2141-4a87-9ead-a89c861476e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.775727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193840-2141-4a87-9ead-a89c861476e3 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.776146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193840-2141-4a87-9ead-a89c861476e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:12.777268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193840-2141-4a87-9ead-a89c861476e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.777967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193840-2141-4a87-9ead-a89c861476e3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.795536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 371/739] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:12.795536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bae0c9-f0b0-4612-990e-4a1cb49b9790 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.795536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bae0c9-f0b0-4612-990e-4a1cb49b9790 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.795536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bae0c9-f0b0-4612-990e-4a1cb49b9790 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.795536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 372/740] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:12.795536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bae0c9-f0b0-4612-990e-4a1cb49b9790 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.839567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:12.862031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec014e-32f2-497e-a8b3-8cd3ba06b1ee None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.898940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.899571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.899855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.900131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:12.911533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:12.912411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bae0c9-f0b0-4612-990e-4a1cb49b9790 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:12.913084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:12.930472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:12.931450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:12.932324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:12.932324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:12.932324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:12.932324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 369/741] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:12.933353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:12.954017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:12.954619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:12.954898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:12.955166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.016098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:13.029031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.030503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.040507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573174df-7c9c-4c19-91ef-4a55de201c41 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.082511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:13.088308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16f05c-6484-4ef9-a8ea-1f0ab8acc897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.091987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f05c-6484-4ef9-a8ea-1f0ab8acc897 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.091987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f05c-6484-4ef9-a8ea-1f0ab8acc897 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.091987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f05c-6484-4ef9-a8ea-1f0ab8acc897 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.097647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16f05c-6484-4ef9-a8ea-1f0ab8acc897 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.099129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16f05c-6484-4ef9-a8ea-1f0ab8acc897 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-4d16f05c-6484-4ef9-a8ea-1f0ab8acc897 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:13.110335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:13.111260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:13.112202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 370/742] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:13.112799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe9314-d281-4b08-9b6f-c114b9786ca7 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.112799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe9314-d281-4b08-9b6f-c114b9786ca7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.113071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe9314-d281-4b08-9b6f-c114b9786ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.132969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-9fbe9314-d281-4b08-9b6f-c114b9786ca7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:13.134185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9fbe9314-d281-4b08-9b6f-c114b9786ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:13.159721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 373/743] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 13:42:13.160564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.203468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 371/744] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 13:42:13.204279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe9314-d281-4b08-9b6f-c114b9786ca7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.257007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:13.273126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f808a-c8e1-423d-ac8d-c60e64e29848 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.315541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe9314-d281-4b08-9b6f-c114b9786ca7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.318938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.319554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.319828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.320092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.320739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:13.321032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.322408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.339907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 374/745] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:13.339907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.341327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:13.356751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.357461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.357737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.358032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.358677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.360197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.438219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7830a7-2926-4784-b1a7-da5aca1631d3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.472049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.472654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.472927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.473198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.473976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:13.474282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.475691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.483807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 372/746] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:13 2022] POST /identity/v3/services => generated 251 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:13.484662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.500748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 375/747] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:42:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:13.501791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.509696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:13.613712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e912b-43ce-4724-998b-8796ad10c1a4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.632133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ccc36-77e1-4952-9d0c-cd6706e8bf44 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.654099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.654744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.655022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.655302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.666733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.668205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.674966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.676029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.676313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.676582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:13.683566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:13.684565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:13.685465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:13.685465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:13.685465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:13.685465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 373/748] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:13.685694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.689868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.692574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:13.708563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:13.709542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:13.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:13.711575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 376/749] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:42:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:13.711644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.717731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:13.784126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:13.796252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202054f8-d0fa-4cd8-bb09-a500675a7788 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.832074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d5cd3-59fe-4c2f-88cc-8923c9c8b00f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:13.833655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.834577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.834857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.835177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.835837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.837199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.866627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:13.867431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:13.867710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:13.867979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:13.878960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:13.880401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:13.894931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:13.896165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:13.897501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:13.897501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:13.897501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:13.900764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 377/750] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:13.901454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:13.917951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 374/751] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:13 2022] POST /identity/v3/services => generated 240 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:13.926951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.034793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e740f-ab90-4994-9776-d8aa24b76fca None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.050808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.060309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.071481 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090b3e-29b3-4d26-a3f0-2005b99855ca None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.081403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.081403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.081656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.082474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.083631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:14.084074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.085545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.102587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 378/752] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:14.102910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.129930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.129930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.129930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.129930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.130369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:14.130369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.130369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.143020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 375/753] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Sep 1 13:42:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:14.144094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.230811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6f65-16f5-401a-a86d-ffcee37c4b34 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.252975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2236c7-a90b-4d52-a052-1af2306d060e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.269262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.270024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.270310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.270582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.271373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:14.271684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.273044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.286420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 379/754] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:14.287477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.295460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.296237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.297169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.297534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.304735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.306324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:14.321210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:14.322645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:14.323637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:14.324284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 376/755] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:14.325810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.326040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.329445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.396316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ceb0f1-2468-4274-a3d5-876f21c83543 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.432100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a475a-c961-4e31-98b5-96fc7c427c28 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.434614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.435615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.435615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.435892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.476661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.477461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.477873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.478297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.487346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.489572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:14.504463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:14.505457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:14.506374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:14.510622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 377/756] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:14.511449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.512473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.514015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.552338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.589428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.632669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 380/757] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:42:14.633791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.641527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c305a4-73f1-4185-8505-83d0e865a165 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.679178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.679964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.680301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.680633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.681535 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:14.681933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.683480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.699009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 378/758] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:14.700076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.743443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3545d2-c086-4ba7-b95a-a13d15b04d65 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.780241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.781089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.781465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.781872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.782773 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:14.783178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.784870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.787780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.798558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 381/759] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:14.799797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.809152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3057c-9bc9-4fae-a237-4e7727110499 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.835603 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:14.848011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.848946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.849401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.849969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.861584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.863472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:14.881029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:14.882241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:14.883323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:14.883323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:14.883323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:14.885183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 379/760] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:14.886169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:14.916667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944b3d2-886d-4161-a0f9-46b1b78d2503 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:14.954602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:14.955366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:14.955817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:14.956414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:14.957672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=0f29e56690b343269681dfd8caffe001) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:14.958105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:14.959842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:14.966254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 382/761] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/role_assignments?user.id=0f29e56690b343269681dfd8caffe001&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:14.967197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:15.001621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95455-17f8-404d-9663-82bee51848ee None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:15.036940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:15.042806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:15.043693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.044114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.044525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/19a91e958d514c0d8a61289bbe3ea63d/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.047471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=19a91e958d514c0d8a61289bbe3ea63d, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:15.049292 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:15.085298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c2956-728c-41e4-9090-6d9a0f27f613 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:15.124213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:15.125142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.125520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.125942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.126711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:15.127121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:15.128626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:15.143657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 383/762] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:15.147595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 380/763] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:14 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/19a91e958d514c0d8a61289bbe3ea63d/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:15.149045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:15.263726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f534e5-728c-461a-89ad-188f59f075b5 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:15.287003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:15.302034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:15.302912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.303398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.303788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.311497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:15.313139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:15.325187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:15.326348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:15.327417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:15.329614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 384/764] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:15.335481 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:15.387558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:15.465238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45457c35-7a32-4604-8ea5-4597eaa2ded2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:15.504223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97effb8-6feb-46d4-a398-b7d85f26d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:15.504999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97effb8-6feb-46d4-a398-b7d85f26d4af None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.505306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97effb8-6feb-46d4-a398-b7d85f26d4af None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.505585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97effb8-6feb-46d4-a398-b7d85f26d4af None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.506340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97effb8-6feb-46d4-a398-b7d85f26d4af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:15.506645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97effb8-6feb-46d4-a398-b7d85f26d4af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:15.508075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97effb8-6feb-46d4-a398-b7d85f26d4af None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:15.522062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 381/765] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:15.527872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:15.636310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a7a42-61ea-4337-b8c1-260a3de496d5 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.636756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a7a42-61ea-4337-b8c1-260a3de496d5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.637130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a7a42-61ea-4337-b8c1-260a3de496d5 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.638858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 382/766] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:15.644238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038f3a4-e7e3-4133-b72e-4151d398b707 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.644638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038f3a4-e7e3-4133-b72e-4151d398b707 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.645009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038f3a4-e7e3-4133-b72e-4151d398b707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.668286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166d823-a710-4db3-9a5f-33ea92cf84db None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:15.693573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c038f3a4-e7e3-4133-b72e-4151d398b707 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:15.694967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c038f3a4-e7e3-4133-b72e-4151d398b707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:15.704328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:15.705171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:15.705599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:15.706042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:15.706915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=2dbc06c0703142ba98ffeb72280e1e63) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:15.707366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:15.708949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:15.715109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 385/767] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:15 2022] GET /identity/v3/role_assignments?user.id=2dbc06c0703142ba98ffeb72280e1e63&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:15.853188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 383/768] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:16.227731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:16.283823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:16.340895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61115ccf-4aa0-4dcb-8e63-17115efa7eb2 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:16.341867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61115ccf-4aa0-4dcb-8e63-17115efa7eb2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:16.342361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61115ccf-4aa0-4dcb-8e63-17115efa7eb2 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:16.345170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 384/769] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:16.348552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8887d3-db6b-4ef7-bc37-065e85935944 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:16.352404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695c8af-b9ab-47c7-bc73-47c4b616b9e3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:16.352531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695c8af-b9ab-47c7-bc73-47c4b616b9e3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:16.352720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695c8af-b9ab-47c7-bc73-47c4b616b9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:16.392009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-8695c8af-b9ab-47c7-bc73-47c4b616b9e3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:16.392933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:16.393855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:16.394331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:16.394799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:16.395575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:16.397126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:16.398969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8695c8af-b9ab-47c7-bc73-47c4b616b9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:16.410028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:16.411030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:16.411865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:16.421713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 386/770] 10.222.0.61 () {56 vars in 1129 bytes} [Thu Sep 1 13:42:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:16.427440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:16.450143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:16.544970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 385/771] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:16.623493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c780b3f-40f3-4be8-b081-d772d8526910 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:16.638292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:16.675407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:16.676173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:16.676769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:16.677048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:16.677692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:16.678034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:16.679582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:16.688163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 387/772] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Sep 1 13:42:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:16.693696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695c8af-b9ab-47c7-bc73-47c4b616b9e3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:16.821144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695c8af-b9ab-47c7-bc73-47c4b616b9e3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:16.869443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:16.870081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:16.870359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:16.870631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:16.871264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:16.871562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:16.872922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:16.878150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 386/773] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Sep 1 13:42:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:16.883680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:16.989666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:17.034466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283bdf3-8c2f-4f10-82dd-f95da7ce2ec9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:17.062948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:17.110100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb63a4f-b931-4574-a29b-f06d33e7624c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:17.128157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:17.128775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.129051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.129316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.130053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] RBAC: Adding query filter params (interface=public, service_id=c8e309715fcc479aa67c87dd2c64df13, region_id=RegionOne) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:17.130361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:17.131714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:17.153270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 388/774] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:42:16 2022] GET /identity/v3/endpoints?service_id=c8e309715fcc479aa67c87dd2c64df13&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:17.159429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:17.178213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10083d9f-9b32-43fb-94b2-8aa70e5ad725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:17.179386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10083d9f-9b32-43fb-94b2-8aa70e5ad725 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.180162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10083d9f-9b32-43fb-94b2-8aa70e5ad725 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.180666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10083d9f-9b32-43fb-94b2-8aa70e5ad725 None admin] PATH_INFO: `/v3/services/network` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.181486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10083d9f-9b32-43fb-94b2-8aa70e5ad725 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:17.183651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10083d9f-9b32-43fb-94b2-8aa70e5ad725 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:17.197127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-10083d9f-9b32-43fb-94b2-8aa70e5ad725 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:17.200014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:17.201009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:17.201952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:17.203873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 387/775] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:16 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:17.206147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69b86-0796-4ddd-beff-91c4fdb7e745 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.207249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69b86-0796-4ddd-beff-91c4fdb7e745 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.207970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69b86-0796-4ddd-beff-91c4fdb7e745 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.213710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 388/776] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:17.215853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e5db-83f2-4c14-893c-a0ef6cfa1a06 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.216296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e5db-83f2-4c14-893c-a0ef6cfa1a06 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.216888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e5db-83f2-4c14-893c-a0ef6cfa1a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.240347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-87a5e5db-83f2-4c14-893c-a0ef6cfa1a06 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:17.242745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-87a5e5db-83f2-4c14-893c-a0ef6cfa1a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:17.290350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15560094-5224-4340-8d87-c3f23616e3ad None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:17.341089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:17.341699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.342014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.342287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.342713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 389/777] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:17.343246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:17.344375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:17.344642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5e5db-83f2-4c14-893c-a0ef6cfa1a06 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:17.361620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:17.369820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 389/778] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:17 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:17.516022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5e5db-83f2-4c14-893c-a0ef6cfa1a06 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:17.559498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:17.560115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.560395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.560667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.561301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] RBAC: Adding query filter params (name=network) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:17.561601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:17.563564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:17.568597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 390/779] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:17.573986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:17.681917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01c87f-df72-4aed-a19a-8e8ce93556a6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:17.683875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:17.717833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e151851-2346-47d2-a395-4d7e77787566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:17.718970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e151851-2346-47d2-a395-4d7e77787566 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.719250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e151851-2346-47d2-a395-4d7e77787566 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.719525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e151851-2346-47d2-a395-4d7e77787566 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.720169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e151851-2346-47d2-a395-4d7e77787566 None admin] RBAC: Adding query filter params (type=network) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:17.720472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e151851-2346-47d2-a395-4d7e77787566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:17.722806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e151851-2346-47d2-a395-4d7e77787566 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:17.722930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:17.729672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 390/780] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:17.731045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9559ee4-595b-4dcf-8fa9-a8cbca13b0ea None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.731335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9559ee4-595b-4dcf-8fa9-a8cbca13b0ea None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.731610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9559ee4-595b-4dcf-8fa9-a8cbca13b0ea None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.733305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 391/781] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:17.738751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba733ad-7040-46c0-a5bc-52b55c67eeff None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.739041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba733ad-7040-46c0-a5bc-52b55c67eeff None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.739320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba733ad-7040-46c0-a5bc-52b55c67eeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.760539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8ba733ad-7040-46c0-a5bc-52b55c67eeff None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:17.761689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8ba733ad-7040-46c0-a5bc-52b55c67eeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:17.816140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:17.843737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2667b-b8e0-49b1-95ee-fdc58567a441 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:17.885110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:17.886234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.886735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.887207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.899938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:17.906017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:17.907014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:17.907848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:17.910648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 391/782] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:17.911790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:17.948000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 392/783] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:17.949419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d973a-54ab-4f1d-9602-b63b7b335e04 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:17.949708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d973a-54ab-4f1d-9602-b63b7b335e04 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:17.950043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d973a-54ab-4f1d-9602-b63b7b335e04 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:17.951489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 393/784] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:17.952153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d973a-54ab-4f1d-9602-b63b7b335e04 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.047467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0df3-7666-4731-8f1e-38e594ea55bb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.064637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.073537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d973a-54ab-4f1d-9602-b63b7b335e04 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.092175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9f485-9163-42bc-8eaa-42ea4079f3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.093742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9f485-9163-42bc-8eaa-42ea4079f3ba None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.094251 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9f485-9163-42bc-8eaa-42ea4079f3ba None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.094724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9f485-9163-42bc-8eaa-42ea4079f3ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.096169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9f485-9163-42bc-8eaa-42ea4079f3ba None admin] RBAC: Adding query filter params (interface=public, service_id=15f16f666b4e4922b873ba04a1f3a7a9, region_id=RegionOne) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:18.096837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9f485-9163-42bc-8eaa-42ea4079f3ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.098738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9f485-9163-42bc-8eaa-42ea4079f3ba None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:18.105398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.115164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 392/785] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:42:17 2022] GET /identity/v3/endpoints?service_id=15f16f666b4e4922b873ba04a1f3a7a9&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:18.115555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01298e5-045c-4394-bd6e-a105d1f474b8 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.123182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01298e5-045c-4394-bd6e-a105d1f474b8 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.123182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01298e5-045c-4394-bd6e-a105d1f474b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.123361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.123361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.123361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.123361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.123649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:18.123649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.123649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:18.135519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 394/786] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:18.136877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c01298e5-045c-4394-bd6e-a105d1f474b8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:18.137139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.138839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c01298e5-045c-4394-bd6e-a105d1f474b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:18.220449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 393/787] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:18.221918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01298e5-045c-4394-bd6e-a105d1f474b8 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.271284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe6cd5-d0af-4aca-8048-7f44e8346b14 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.316817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.317714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.318021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.318288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.318895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.320802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:18.328073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 395/788] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:18.329277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.350950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01298e5-045c-4394-bd6e-a105d1f474b8 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.376263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.406490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.416060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.416711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.416993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.417267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.431724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.435306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:18.470771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:18.472740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:18.474110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:18.474110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:18.474110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:18.474110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 394/789] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:18.481179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.531556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d88bbe-0ec3-4fbe-b003-67cd6e2905c9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.570187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.571341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.571932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.572533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.585063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:18.591120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:18.592079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:18.592943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:18.596284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 396/790] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:18.597476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.615308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4186111-4afc-4df9-acc3-76cb0c1e1c53 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.656829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.658151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.658151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.658151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.658683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:18.658959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.660436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:18.672877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 395/791] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:18.678165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.722972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e844446-172b-4111-a2c0-941c8d369b80 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.725800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.761231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.762306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.762840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.763320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.775426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:18.780994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:18.781965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:18.783116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:18.786675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 397/792] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:18.788130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e15741-146b-42e2-b324-0524f2d67af3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.789335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.794352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.826483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.827102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.827452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.827727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.828640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:18.828949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.830801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:18.839810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 396/793] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:18.840654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.903660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c96bb-4dbc-4a43-885f-b8a8576bb2aa None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.942622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.943713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.944204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.944683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.946309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cabe7b-2d2a-49fd-a471-f71ee6689f54 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:18.948922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.954597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.956344 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:18.968778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:18.969789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:18.970750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:18.973127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 398/794] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:18.973980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:18.978367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:18.990622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:18.991243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:18.996159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:18.996159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:18.996159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:18.996159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:18.996159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.004059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 397/795] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:19.005033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.096594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdce22-7c2e-4cf4-b8dc-c80680040857 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.110558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1be4e-f884-4aae-b937-c6452b92545d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.136469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.137641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.138179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.138708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.146490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.147101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.147381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.147681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.148464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:19.148715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.150246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.153186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.154918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:19.172508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:19.173534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:19.174868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:19.174868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:19.174868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:19.175052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 398/796] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:19.175052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.176910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 399/797] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:19.182399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.201652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:19.250236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:19.283012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3858fab-c37f-42da-945f-9cb9c00e374a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.297150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603db0-63b3-4d7b-a1f4-324315615dc4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.324439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.325081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.325360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.325631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.340144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.341851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.342835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.344457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.344893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.345661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.346488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:19.346961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.348517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:19.361388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:19.362971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:19.363863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:19.363863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:19.363863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:19.363863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 399/798] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:19.363863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.367949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 400/799] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:19.369120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.470735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:19.493801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4983f-91f4-405d-aa5a-c3e5f4fa6edb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.498972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be494f-e4da-408a-be05-5e83735ebac4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.519289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:19.541493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.542161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.542483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.542758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.543381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.543513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:19.543679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.544434 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.544873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.545030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.545416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.559647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.561327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.562634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 401/800] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:19.563633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:19.580444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:19.581581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:19.582669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:19.582669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:19.582669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:19.585088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 400/801] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:19.586244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.674541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69d339-5b9e-43ee-ab0c-3e129d6ccb09 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.691098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d13626-2186-4494-ae0e-6e00d791e8f0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.712721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.713384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.713619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.713919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.719151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:19.722677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.724191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.727114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0e8a3-1e41-448c-bf25-c22d312e6a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.731674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e8a3-1e41-448c-bf25-c22d312e6a5b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.731674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e8a3-1e41-448c-bf25-c22d312e6a5b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.731674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e8a3-1e41-448c-bf25-c22d312e6a5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.739405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0e8a3-1e41-448c-bf25-c22d312e6a5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.740848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0e8a3-1e41-448c-bf25-c22d312e6a5b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:19.742542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:19.743444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:19.744406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 402/802] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:19.744991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-75b0e8a3-1e41-448c-bf25-c22d312e6a5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:19.759741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:19.760708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:19.761640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:19.763888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 401/803] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:19.765741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f07148-0f58-430b-a72c-8d51eb927d45 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.766050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f07148-0f58-430b-a72c-8d51eb927d45 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.766323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f07148-0f58-430b-a72c-8d51eb927d45 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.767695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 402/804] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:19.770589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f07148-0f58-430b-a72c-8d51eb927d45 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.784907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:19.867860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5ce3-e2e7-48cd-87de-c3b10c07ea8d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.884665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f07148-0f58-430b-a72c-8d51eb927d45 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:19.904783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.906420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.906420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.906685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.907275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:19.907581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.908991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.925435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 403/805] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:19.925435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:19.928055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fe7d1-b6b9-4a20-b6f2-b158026af63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:19.928669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fe7d1-b6b9-4a20-b6f2-b158026af63d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.928947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fe7d1-b6b9-4a20-b6f2-b158026af63d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.929222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fe7d1-b6b9-4a20-b6f2-b158026af63d None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.929999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fe7d1-b6b9-4a20-b6f2-b158026af63d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:19.930308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fe7d1-b6b9-4a20-b6f2-b158026af63d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:19.931745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fe7d1-b6b9-4a20-b6f2-b158026af63d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:19.949789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 403/806] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:19.951086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29d596-5b6d-4267-8373-cc64f1480e8c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:19.951378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29d596-5b6d-4267-8373-cc64f1480e8c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:19.951653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29d596-5b6d-4267-8373-cc64f1480e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:19.970502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-7f29d596-5b6d-4267-8373-cc64f1480e8c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:19.971714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7f29d596-5b6d-4267-8373-cc64f1480e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:20.018730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.033276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.042116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f73f-f087-4e5e-8066-08c6a500ce91 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.050655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 404/807] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:20.051699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29d596-5b6d-4267-8373-cc64f1480e8c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.081748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.082408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.082701 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.083284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.084300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:20.084609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.086224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.115448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 404/808] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Sep 1 13:42:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:20.115448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.175577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29d596-5b6d-4267-8373-cc64f1480e8c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.222874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.223519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.223794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.224063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.231192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.232693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.240103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424adbe-c8fd-4983-aba6-a9ff6126b94d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:20.245012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:20.245921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:20.246890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:20.251150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 405/809] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:20.252298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.287480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.288621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.289348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.290276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.294972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.317284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.369397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c44a3-178a-4214-abb9-fb04e970c21d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.377763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.379204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.406686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.407312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.407595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.407867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.417794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.419238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:20.434898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:20.435868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:20.437405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:20.437405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:20.437405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:20.437405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 406/810] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:20.437628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.480610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 405/811] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:42:20.481456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.524740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.532357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.537390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc30f7-fe8e-4e7e-952f-e58c8dd03cb6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.573387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf477dd1-8bf0-4828-8e6f-c5905cd4b490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.573972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf477dd1-8bf0-4828-8e6f-c5905cd4b490 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.574256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf477dd1-8bf0-4828-8e6f-c5905cd4b490 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.574526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf477dd1-8bf0-4828-8e6f-c5905cd4b490 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.575216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf477dd1-8bf0-4828-8e6f-c5905cd4b490 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:20.575521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf477dd1-8bf0-4828-8e6f-c5905cd4b490 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.576947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf477dd1-8bf0-4828-8e6f-c5905cd4b490 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.583320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c626e-0928-4245-accc-54f0f45f1c17 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.588311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 407/812] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:20.589522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca8fe-848c-4f09-826c-f3ce42a79a64 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.589831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca8fe-848c-4f09-826c-f3ce42a79a64 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.590104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca8fe-848c-4f09-826c-f3ce42a79a64 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.591347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 408/813] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:20.592036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ca8fe-848c-4f09-826c-f3ce42a79a64 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.620669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.621294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.621571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.621881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.622464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.623820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:20.632879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:20.634079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:20.634910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 406/814] 10.222.0.61 () {56 vars in 1129 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:20.635685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.696279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ca8fe-848c-4f09-826c-f3ce42a79a64 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.731995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.732631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.732908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.733177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.734390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:20.734688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.736060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.737849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f0530-5dfd-479e-91bf-22bfa3b4ffda None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.748467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 409/815] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:20.749289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.753634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.761008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.772092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8e9cf-63cc-46d7-9d13-44ff4e3443c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.772712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8e9cf-63cc-46d7-9d13-44ff4e3443c2 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.772990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8e9cf-63cc-46d7-9d13-44ff4e3443c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.773264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8e9cf-63cc-46d7-9d13-44ff4e3443c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.784340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8e9cf-63cc-46d7-9d13-44ff4e3443c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.786166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8e9cf-63cc-46d7-9d13-44ff4e3443c2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-17a8e9cf-63cc-46d7-9d13-44ff4e3443c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:20.801204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:20.807414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:20.808397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:20.808397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:20.808397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:20.808397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 407/816] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:20.808397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a1d33-c812-46a1-af4e-f753fb85933d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.808397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a1d33-c812-46a1-af4e-f753fb85933d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.808397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a1d33-c812-46a1-af4e-f753fb85933d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.821976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-607a1d33-c812-46a1-af4e-f753fb85933d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:20.823193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-607a1d33-c812-46a1-af4e-f753fb85933d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:20.866352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbf855-4c93-4d32-963f-64de3b8442de None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:20.893874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 408/817] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:20.894902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a1d33-c812-46a1-af4e-f753fb85933d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:20.901593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:20.902286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:20.902566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:20.902841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/0f29e56690b343269681dfd8caffe001/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:20.906544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=0f29e56690b343269681dfd8caffe001, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:20.907995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:20.987827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:20.991928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 410/818] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:20 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/0f29e56690b343269681dfd8caffe001/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:20.993098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.000340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a1d33-c812-46a1-af4e-f753fb85933d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.048721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.049950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.050441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.051045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.051797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:21.052107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.053595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.064574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 409/819] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:21.065389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.095663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:21.151051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa28cc0-48a1-494e-8205-bdc276846366 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.181619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb6762-60f2-4ece-820d-ed6705d364de None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.195172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.195967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.196241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.196900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.206919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.208817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.216818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.217436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.217717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.218616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.219268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:21.219649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.221115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:21.222822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:21.223698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:21.224648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:21.225124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 411/820] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:21.225659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.237841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 410/821] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:21.243090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.267968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:21.341799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e896c-1fd4-483a-a814-e4c5be97089f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.354252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791e1dd-f384-4585-9209-b1131cf6576a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.371608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:21.379936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.385398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.385398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.385398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.385398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:21.385398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.385398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.394105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 412/822] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Sep 1 13:42:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:21.394105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.395091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.395792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.396131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.396470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] PATH_INFO: `/v3/services/network` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.397117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.398569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:21.409238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:21.410441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 13:42:21.411536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:21.412447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 411/823] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:21 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:21.413012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.504248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:21.511514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ba3b4-177c-4124-8a74-a8199af6e789 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.520827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3a3a-3131-49b9-82b6-73b6693bd453 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.547382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.548023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.548309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.548592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.549289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:21.549606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.551131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.557559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.558873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.559366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.559844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.564356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 413/824] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:21.565170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.568064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.569612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:21.581810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:21.582888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:21.583875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 412/825] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:21.588540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.628929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:21.686327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41eb96-b174-4366-b922-4892b930b608 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.695983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ddf7b-b474-4bc4-91c2-750dba2cd4c1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.724612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.725217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.725504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.725849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.726453 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.727849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.731434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.732052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.732322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.732596 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.733227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] RBAC: Adding query filter params (name=network) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:21.734072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.735477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.740505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 413/826] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:21.741322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.754018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:21.819947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 414/827] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:21.821034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:21.861040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cbc82-6953-48ce-868a-2ecc2c4523dd None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.925504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:21.926513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:21.927020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:21.927495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/2dbc06c0703142ba98ffeb72280e1e63/roles/61fd046fc53c4a57a1ddd3e4f878622c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:21.945118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=2dbc06c0703142ba98ffeb72280e1e63, role_id=61fd046fc53c4a57a1ddd3e4f878622c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:21.946877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:21.966925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774c007-6bf3-4516-b972-ef590f4a8287 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:21.978041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:22.003601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:22.004215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:22.004487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:22.004761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:22.005493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:22.005828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:22.007413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:22.021273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 415/828] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:22.021978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:22.025940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 414/829] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:21 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/2dbc06c0703142ba98ffeb72280e1e63/roles/61fd046fc53c4a57a1ddd3e4f878622c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:22.030073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:22.118894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:22.158931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf60b-4309-40da-8f18-88897421b05a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:22.178822 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68238d-38b8-4329-aa19-6938efa8ca02 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:22.199450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:22.200452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:22.200926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:22.201410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:22.202865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] RBAC: Adding query filter params (type=network) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:22.203364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:22.204994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:22.210916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 416/830] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:22.216518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:22.220814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ab449-0a09-4906-bf65-acad54f7edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:22.221716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ab449-0a09-4906-bf65-acad54f7edde None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:22.226301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ab449-0a09-4906-bf65-acad54f7edde None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:22.227789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ab449-0a09-4906-bf65-acad54f7edde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:22.228873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ab449-0a09-4906-bf65-acad54f7edde None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=19a91e958d514c0d8a61289bbe3ea63d) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:22.230909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ab449-0a09-4906-bf65-acad54f7edde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:22.239115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ab449-0a09-4906-bf65-acad54f7edde None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:22.255954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 415/831] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:22 2022] GET /identity/v3/role_assignments?user.id=19a91e958d514c0d8a61289bbe3ea63d&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:22.275482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:22.364312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d14e64-17b7-436f-8228-edf43359b61a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:22.403643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd9815-f961-4e94-9897-3ebfe17b5915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:22.404281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9815-f961-4e94-9897-3ebfe17b5915 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:22.404527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9815-f961-4e94-9897-3ebfe17b5915 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:22.404801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9815-f961-4e94-9897-3ebfe17b5915 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:22.405519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd9815-f961-4e94-9897-3ebfe17b5915 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:22.406970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd9815-f961-4e94-9897-3ebfe17b5915 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:22.496142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 417/832] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:23.057227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4742804-2eeb-413c-96fe-9b40f730e12e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:23.057794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4742804-2eeb-413c-96fe-9b40f730e12e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:23.057794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4742804-2eeb-413c-96fe-9b40f730e12e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:23.059379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 416/833] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:23.064799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d64067-716b-4dd8-8033-6122fadc72a2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:23.065087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d64067-716b-4dd8-8033-6122fadc72a2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:23.065362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d64067-716b-4dd8-8033-6122fadc72a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:23.087951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-46d64067-716b-4dd8-8033-6122fadc72a2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:23.089148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-46d64067-716b-4dd8-8033-6122fadc72a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:23.171567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 418/834] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:23.460294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4742804-2eeb-413c-96fe-9b40f730e12e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:23.564854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4742804-2eeb-413c-96fe-9b40f730e12e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:23.583588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:23.600584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:23.601211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:23.601486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:23.601761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:23.612013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:23.616735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:23.618286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:23.619499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 417/835] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:23.624350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d64067-716b-4dd8-8033-6122fadc72a2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:23.641906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:23.734758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d64067-716b-4dd8-8033-6122fadc72a2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:23.771199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:23.772051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:23.772512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:23.772947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:23.773758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:23.774306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:23.775827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:23.785051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 419/836] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:23.793486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:23.898366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e6bf0-5d7d-4ca5-9e36-c3fdfd15a290 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:23.941605 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9a94-6bab-491b-9d40-18e6b83d6888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:23.942275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9a94-6bab-491b-9d40-18e6b83d6888 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:23.942551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9a94-6bab-491b-9d40-18e6b83d6888 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:23.942816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9a94-6bab-491b-9d40-18e6b83d6888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:23.953942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc9a94-6bab-491b-9d40-18e6b83d6888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:23.955385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc9a94-6bab-491b-9d40-18e6b83d6888 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-c6fc9a94-6bab-491b-9d40-18e6b83d6888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:23.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:23.971923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:23.972990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:23.972990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:23.972990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:23.972990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 418/837] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:23.978195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.063009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:24.079747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5870-0261-4691-a302-5e3a48d35588 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.080297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5870-0261-4691-a302-5e3a48d35588 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.080297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5870-0261-4691-a302-5e3a48d35588 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.081889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 419/838] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:24.090289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db035e42-1280-4bd7-9c6d-51c26a197265 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.090698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db035e42-1280-4bd7-9c6d-51c26a197265 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.091082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db035e42-1280-4bd7-9c6d-51c26a197265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.094152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e448504-f27b-4990-b9ee-b274afcc4b4f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.110956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-db035e42-1280-4bd7-9c6d-51c26a197265 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:24.112304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-db035e42-1280-4bd7-9c6d-51c26a197265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:24.137017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.137810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.138203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.138743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.140168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:24.140474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.142629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:24.143629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.162354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 420/839] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:24.167395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.194312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 420/840] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:24.263146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a266b-3f79-4593-9f4c-952b188c7014 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.263793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a266b-3f79-4593-9f4c-952b188c7014 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.264298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a266b-3f79-4593-9f4c-952b188c7014 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.266220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 421/841] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:24.271399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f5f2-82bf-40f0-8df0-555d5f1434c6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.271666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f5f2-82bf-40f0-8df0-555d5f1434c6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.271936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f5f2-82bf-40f0-8df0-555d5f1434c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.292457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fd6c2-52eb-44f0-8704-66c3e06b641b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.296929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ce43f5f2-82bf-40f0-8df0-555d5f1434c6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:24.298191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ce43f5f2-82bf-40f0-8df0-555d5f1434c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:24.336892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.337887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.338307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.338766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.342334 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:24.348841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.350805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:24.362969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:24.364051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 13:42:24.364992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:24.369197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 421/842] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:24.370338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.394175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 422/843] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:24.470776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f5f2-82bf-40f0-8df0-555d5f1434c6 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.488311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dcce4-efa8-4ae4-b310-5c5e2f9b2246 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.536138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846d489-4a02-4414-b514-4ec6f9698afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.536754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846d489-4a02-4414-b514-4ec6f9698afc None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.537182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846d489-4a02-4414-b514-4ec6f9698afc None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.537463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846d489-4a02-4414-b514-4ec6f9698afc None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.538813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846d489-4a02-4414-b514-4ec6f9698afc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:24.539133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846d489-4a02-4414-b514-4ec6f9698afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.540570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846d489-4a02-4414-b514-4ec6f9698afc None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.569449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 422/844] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:24.569449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d337-9049-4bc3-8c5b-7ba9c4bd69ca None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.569449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d337-9049-4bc3-8c5b-7ba9c4bd69ca None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.569449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2d337-9049-4bc3-8c5b-7ba9c4bd69ca None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.569855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 423/845] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:24.570735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2d337-9049-4bc3-8c5b-7ba9c4bd69ca None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.570992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:24.578624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f5f2-82bf-40f0-8df0-555d5f1434c6 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.613873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d590-a06e-4a6e-a16f-5981b4c0b94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.614510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d590-a06e-4a6e-a16f-5981b4c0b94d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.614793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d590-a06e-4a6e-a16f-5981b4c0b94d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.615067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d590-a06e-4a6e-a16f-5981b4c0b94d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.615656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d590-a06e-4a6e-a16f-5981b4c0b94d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.617299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d590-a06e-4a6e-a16f-5981b4c0b94d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-1f76d590-a06e-4a6e-a16f-5981b4c0b94d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:24.627865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:24.628886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:24.629741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 423/846] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:24.631104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddbab6-2bdc-4edc-9438-431914c2ae68 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.631201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddbab6-2bdc-4edc-9438-431914c2ae68 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.631451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddbab6-2bdc-4edc-9438-431914c2ae68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.649421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:24.651129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d5ddbab6-2bdc-4edc-9438-431914c2ae68 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:24.652328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d5ddbab6-2bdc-4edc-9438-431914c2ae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:24.695869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2d337-9049-4bc3-8c5b-7ba9c4bd69ca None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.721160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 424/847] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:24.721958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddbab6-2bdc-4edc-9438-431914c2ae68 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.738989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.739595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.739867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.740136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.813415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:24.819235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.820937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.829247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddbab6-2bdc-4edc-9438-431914c2ae68 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:24.868798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:24.869501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:24.869817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:24.870098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:24.871012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:24.871416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:24.873133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:24.883701 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 425/848] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:24.884528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.887130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:24.936667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 424/849] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:42:24.937404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:24.996576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1435b-6166-49c4-ae55-38487a5cc209 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.031945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.032614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.032891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.033308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.043783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:25.044557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396793f-cfa0-446e-932b-40ca1deccf4d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:25.048659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:25.049808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:25.050861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 426/850] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:25.051722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.062186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.082028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.082636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.082912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.083177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.083793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:25.084088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.085451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.089824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 425/851] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:25.090576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.112638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.170226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f61895-44f6-4ce4-9511-2cc451aba78a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.205305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cecf3-fe53-4475-a44f-134568cdb3d3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.210750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.211432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.211674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.211940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.222697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.224155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:25.240526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:25.241580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:25.242779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:25.242779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:25.242779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:25.242779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 427/852] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:25.242779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.250345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.250933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.251214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.251486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.252081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.253473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:25.262821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:25.263857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:25.270775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 426/853] 10.222.0.61 () {56 vars in 1132 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:25.271749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.315250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.361475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e36e8d-18ed-4514-838c-96caa0e9d7f8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.377074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.395766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07247ba-b072-440a-b3ee-c35f610d4f82 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.398565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.399180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.399460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.399729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.400399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:25.400702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.402107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.411292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 428/854] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:25.412114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.440956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.441837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.442268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.442692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.443500 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:25.443805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.445248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.461550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 427/855] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:25.461550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.525684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066cb42-3219-4dd6-b9bc-2c689e9df053 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.553509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.567520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.568245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.568526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.568947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.569591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:25.569948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.571492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.575311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a6647-e9e5-45e6-a607-83a8b9ba2319 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.581625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 429/856] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:25.582707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.611874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.612912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.613441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.613976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.628019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.630101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.632619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:25.649720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:25.652730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:25.654923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:25.654923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:25.654923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:25.657973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 428/857] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:25.662087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.700829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c3c8f-76d9-4878-a8f6-f25db242114d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.736039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.736852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.737232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.737607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.744943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.746537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:25.758106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:25.759195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:25.760287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:25.761892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 430/858] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:25.767751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.777873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddbf6b-8e42-4a92-bc18-b620037b11cd None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.816380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.816916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.817820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.818101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.818387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.819168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:25.819475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.821150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.826188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 429/859] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:25.828306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.887004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c337c5-5b9d-4a2e-b1f1-7074532aece2 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:25.918018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:25.937582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:25.938321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:25.938665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:25.939003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:25.940425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:25.940782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:25.942361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:25.962157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 431/860] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:25.969001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:25.980594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390670f-ecf7-4313-baa8-7b32d8f1b011 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:26.021164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:26.022801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.023089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.023361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.024077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:26.024432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:26.029923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:26.042530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 430/861] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:26.048495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:26.114229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81935e69-ed66-488b-b20a-b2ea6b3d5065 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:26.123520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:26.152930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:26.153884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.154309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.154725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.163001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:26.164743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:26.177229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5825-b616-4960-8aae-5d85a4a0bb36 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:26.178824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:26.180061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:26.181001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:26.183550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 432/862] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:42:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:26.189611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:26.199721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:26.218926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:26.219570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.220119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.220229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.220969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=0f29e56690b343269681dfd8caffe001) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:26.221324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:26.222869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:26.229804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 431/863] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:26 2022] GET /identity/v3/role_assignments?user.id=0f29e56690b343269681dfd8caffe001&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:26.314236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20088f-b2f0-419d-9773-85b89e366034 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:26.353922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:26.354764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.355140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.355510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.356363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:26.356768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:26.358886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:26.380360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 433/864] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Sep 1 13:42:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:26.385921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:26.502734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c9f56-e0c9-4687-b659-36c3a25a51eb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:26.538838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:26.539503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.539784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.540061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.601580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:26.623387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:26.625935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:26.735028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 432/865] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:42:26.745804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:26.806306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:26.860325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f913f6a-80ab-4edf-b4c4-cfabec1c2593 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:26.898058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023c96-68c1-4d85-812a-b3f2f371d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:26.898859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023c96-68c1-4d85-812a-b3f2f371d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:26.899238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023c96-68c1-4d85-812a-b3f2f371d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:26.899615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023c96-68c1-4d85-812a-b3f2f371d5d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:26.911304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023c96-68c1-4d85-812a-b3f2f371d5d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:26.912957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023c96-68c1-4d85-812a-b3f2f371d5d1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-68023c96-68c1-4d85-812a-b3f2f371d5d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:26.929015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:26.930242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:26.931335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:26.931335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:26.931335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:26.937098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 434/866] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:26.938516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:27.028373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c30e0-9ac7-4825-b234-94c555a5452f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.028662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c30e0-9ac7-4825-b234-94c555a5452f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.028938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c30e0-9ac7-4825-b234-94c555a5452f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.029923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:27.030985 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 435/867] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:27.036607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e08a6-d0cc-4f2c-a78b-512d56a3757d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.036896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e08a6-d0cc-4f2c-a78b-512d56a3757d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.037176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e08a6-d0cc-4f2c-a78b-512d56a3757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.049873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9cb5b-ba3a-4b6b-a5d3-9f28e6a50b66 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:27.056634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a33e08a6-d0cc-4f2c-a78b-512d56a3757d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:27.057872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a33e08a6-d0cc-4f2c-a78b-512d56a3757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:27.085488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:27.086163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.086452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.086734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.087384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:27.087692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:27.089331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:27.102062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 433/868] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:27.107642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:27.138480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 436/869] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:27.214585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da122-2d6a-4b02-a381-87b3693f4d94 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:27.251884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:27.252531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.252810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.253084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.260272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:27.261733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:27.273682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:27.274623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:27.275533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 434/870] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:27.281230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e08a6-d0cc-4f2c-a78b-512d56a3757d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:27.292135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:27.394164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e08a6-d0cc-4f2c-a78b-512d56a3757d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:27.423312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:27.427119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:27.435942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:27.440279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.440279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.440279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.440279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:27.440279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:27.440990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:27.458326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 437/871] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:27.464265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:27.537338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:27.538141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf2da9-224f-4e31-a447-2db57a0fc5cb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:27.575169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b91e1-145d-4fc7-b647-cd4776c0b924 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:27.575548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d6732-5cbb-419b-b861-31b98e87293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:27.576372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6732-5cbb-419b-b861-31b98e87293e None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.576972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6732-5cbb-419b-b861-31b98e87293e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.577578 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d6732-5cbb-419b-b861-31b98e87293e None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.578998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d6732-5cbb-419b-b861-31b98e87293e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:27.580759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d6732-5cbb-419b-b861-31b98e87293e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:27.612946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562d01d-366a-4159-b4df-dd9a7aa3e271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:27.613603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d01d-366a-4159-b4df-dd9a7aa3e271 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.613914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d01d-366a-4159-b4df-dd9a7aa3e271 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.614229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d01d-366a-4159-b4df-dd9a7aa3e271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.614960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562d01d-366a-4159-b4df-dd9a7aa3e271 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=2dbc06c0703142ba98ffeb72280e1e63) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:27.615374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562d01d-366a-4159-b4df-dd9a7aa3e271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:27.617291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562d01d-366a-4159-b4df-dd9a7aa3e271 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:27.624320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 438/872] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:27 2022] GET /identity/v3/role_assignments?user.id=2dbc06c0703142ba98ffeb72280e1e63&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:27.655293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 435/873] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:27 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:27.845520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fd040-02c1-4631-9762-511810b3f553 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.846042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fd040-02c1-4631-9762-511810b3f553 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.846467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fd040-02c1-4631-9762-511810b3f553 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.849348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 439/874] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:27.854805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f4fc1-ff33-47bf-a298-33edf321e73e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:27.855303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f4fc1-ff33-47bf-a298-33edf321e73e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:27.855630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f4fc1-ff33-47bf-a298-33edf321e73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:27.880467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f90f4fc1-ff33-47bf-a298-33edf321e73e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:27.881676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f90f4fc1-ff33-47bf-a298-33edf321e73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:27.944205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:27.964076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 436/875] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:27 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:28.245671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fd040-02c1-4631-9762-511810b3f553 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:28.354115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fd040-02c1-4631-9762-511810b3f553 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:28.386441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:28.392685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:28.393337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:28.393642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:28.393912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:28.402019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:28.402019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:28.402217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ef003-e57f-4621-9a5f-1fcc1754e2ff None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:28.402217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ef003-e57f-4621-9a5f-1fcc1754e2ff None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:28.402217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ef003-e57f-4621-9a5f-1fcc1754e2ff None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:28.402217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 437/876] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:28.405914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72800b6e-384b-4eb0-a0e8-f4b0ca023ce2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:28.406198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72800b6e-384b-4eb0-a0e8-f4b0ca023ce2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:28.406468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72800b6e-384b-4eb0-a0e8-f4b0ca023ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:28.426173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-72800b6e-384b-4eb0-a0e8-f4b0ca023ce2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:28.427363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-72800b6e-384b-4eb0-a0e8-f4b0ca023ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:28.497726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 438/877] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:28 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:28.521251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 440/878] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:28 2022] POST /identity/v3/services => generated 269 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:28.773464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72800b6e-384b-4eb0-a0e8-f4b0ca023ce2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:28.880201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72800b6e-384b-4eb0-a0e8-f4b0ca023ce2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:28.898830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:28.922033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:28.922663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:28.922934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:28.923199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:28.923770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:28.925135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:28.934027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:28.935070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:28.935971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:28.940045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 439/879] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:28.942874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:29.054112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2a290-d460-4edc-9e69-2a3bb2ea3b82 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:29.089963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f7919-8216-4f1e-a80d-c967fedf904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:29.090950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f7919-8216-4f1e-a80d-c967fedf904a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.091453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f7919-8216-4f1e-a80d-c967fedf904a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.091928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f7919-8216-4f1e-a80d-c967fedf904a None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.093371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f7919-8216-4f1e-a80d-c967fedf904a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:29.093913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f7919-8216-4f1e-a80d-c967fedf904a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:29.095668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f7919-8216-4f1e-a80d-c967fedf904a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:29.101338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 441/880] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:29.107015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:29.213250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab11b6-7463-4d5f-b3b6-3b6d09783924 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:29.249372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c353e58-67c7-48df-9415-bd0be1c20be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:29.250018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c353e58-67c7-48df-9415-bd0be1c20be0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.250299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c353e58-67c7-48df-9415-bd0be1c20be0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.250574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c353e58-67c7-48df-9415-bd0be1c20be0 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.251208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c353e58-67c7-48df-9415-bd0be1c20be0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:29.251511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c353e58-67c7-48df-9415-bd0be1c20be0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:29.252890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c353e58-67c7-48df-9415-bd0be1c20be0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:29.261908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 440/881] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:29.709727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65389b59-6186-4250-bd80-b788cf431585 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.710791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65389b59-6186-4250-bd80-b788cf431585 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.711287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65389b59-6186-4250-bd80-b788cf431585 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.713138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 442/882] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:29.716668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973b81-57a2-4ecb-ad9c-7076e9b0b05f None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.716956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973b81-57a2-4ecb-ad9c-7076e9b0b05f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.717235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973b81-57a2-4ecb-ad9c-7076e9b0b05f None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.718977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 441/883] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:29.719255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc50f1-52e5-4dfc-9889-f8505456c3eb None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.719766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc50f1-52e5-4dfc-9889-f8505456c3eb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.720246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc50f1-52e5-4dfc-9889-f8505456c3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.723799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633946-e2a5-489a-9389-4f7b6a969f6b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:29.724086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633946-e2a5-489a-9389-4f7b6a969f6b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:29.724365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633946-e2a5-489a-9389-4f7b6a969f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:29.739789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-82fc50f1-52e5-4dfc-9889-f8505456c3eb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:29.741304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-82fc50f1-52e5-4dfc-9889-f8505456c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:29.745598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-67633946-e2a5-489a-9389-4f7b6a969f6b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:29.746856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-67633946-e2a5-489a-9389-4f7b6a969f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:29.793589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:29.799328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:29.823885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 443/884] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:29.834195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 442/885] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:30.091329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc50f1-52e5-4dfc-9889-f8505456c3eb None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.125364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67633946-e2a5-489a-9389-4f7b6a969f6b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.219670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc50f1-52e5-4dfc-9889-f8505456c3eb None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.255333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67633946-e2a5-489a-9389-4f7b6a969f6b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.263439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.264991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.265491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.266011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.267342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.269643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:30.280702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:30.281738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:30.282597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:30.288516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 444/886] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:30.290719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.307407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.314239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.314239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.314413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.325712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:30.330565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:30.331516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:30.332439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 443/887] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:30.337665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.355870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.379700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.424919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf50479-71f1-4fc4-8254-485e2cca5080 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.467270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.468393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.468735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d0549-6c08-44bd-958f-d58cf4211ab8 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.468845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.469043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.469687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:30.470084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.471673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.482706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 445/888] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:30.484943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.520530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e04ca-76c9-4254-a241-d3a6eb4022c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.521191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e04ca-76c9-4254-a241-d3a6eb4022c5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.521446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e04ca-76c9-4254-a241-d3a6eb4022c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.521720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e04ca-76c9-4254-a241-d3a6eb4022c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.522445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e04ca-76c9-4254-a241-d3a6eb4022c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:30.522732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e04ca-76c9-4254-a241-d3a6eb4022c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.524165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e04ca-76c9-4254-a241-d3a6eb4022c5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.537594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 444/889] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:30.538769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a942e-df22-448d-b83b-d64f618bec00 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.539057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a942e-df22-448d-b83b-d64f618bec00 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.539333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a942e-df22-448d-b83b-d64f618bec00 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.541821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 445/890] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:30.541821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a942e-df22-448d-b83b-d64f618bec00 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.601268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163bfde-bda1-458d-b679-6ae93a7e1401 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.615846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.638733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ed282-05ba-49b2-8f07-c30322f300ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.639449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed282-05ba-49b2-8f07-c30322f300ed None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.639737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed282-05ba-49b2-8f07-c30322f300ed None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.640004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ed282-05ba-49b2-8f07-c30322f300ed None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.640644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ed282-05ba-49b2-8f07-c30322f300ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:30.641419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ed282-05ba-49b2-8f07-c30322f300ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.642821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ed282-05ba-49b2-8f07-c30322f300ed None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.647523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.648723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 446/891] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:30.649836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429afcd1-b089-47b9-9729-a4c6ba1d23d0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.650134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429afcd1-b089-47b9-9729-a4c6ba1d23d0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.650395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429afcd1-b089-47b9-9729-a4c6ba1d23d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.660895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a942e-df22-448d-b83b-d64f618bec00 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.670007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-429afcd1-b089-47b9-9729-a4c6ba1d23d0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:30.671744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-429afcd1-b089-47b9-9729-a4c6ba1d23d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:30.703154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.703787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.704064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.704334 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.714831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.716239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:30.731680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:30.732798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:30.734327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:30.734327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:30.734327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:30.734327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 446/892] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:30.734327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.750560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 447/893] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:30.751424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429afcd1-b089-47b9-9729-a4c6ba1d23d0 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.866749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4776198-a51f-4e84-af06-d9d2d86ded2b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.880933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429afcd1-b089-47b9-9729-a4c6ba1d23d0 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:30.889897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:30.919520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.920694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.921020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.921320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.922032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] RBAC: Adding query filter params (interface=public, service_id=2f036deca8df4153ae85d3edcc64fd1a, region_id=RegionOne) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:30.922341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.923710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.927125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:30.927737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:30.928012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:30.928281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:30.928928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:30.929228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:30.930681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:30.939673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 447/894] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/endpoints?service_id=2f036deca8df4153ae85d3edcc64fd1a&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:30.942071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.946569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 448/895] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:30.955183 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:30.959738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.063637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034c5c-20e1-480a-9c07-76792a00a1ef None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.081221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e8b84-7589-496a-9b35-2811abb354f9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.100744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.101459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.101742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.102048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.102687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.104033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.118064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 448/896] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:42:31.118505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.119167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.119443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.119714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.120331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.130706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.132313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:31.149145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:31.150471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:31.151485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 449/897] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:42:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:31.156582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.197662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.235207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.266079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb77b36-fca1-4c1a-b865-fa6b0eff2320 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.292190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90697-afe5-491b-a726-4ff335635139 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.306213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b53d1d-8c19-48c1-ac1a-9d9e38a233d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.306978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b53d1d-8c19-48c1-ac1a-9d9e38a233d1 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.307260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b53d1d-8c19-48c1-ac1a-9d9e38a233d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.310965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b53d1d-8c19-48c1-ac1a-9d9e38a233d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.310965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b53d1d-8c19-48c1-ac1a-9d9e38a233d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.310965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b53d1d-8c19-48c1-ac1a-9d9e38a233d1 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-c7b53d1d-8c19-48c1-ac1a-9d9e38a233d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:31.320220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:31.321339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:31.322282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 449/898] 10.222.0.61 () {56 vars in 1132 bytes} [Thu Sep 1 13:42:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:31.324008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb88c9f-7c4f-4cd8-b6a1-88b36d543a56 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.324008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb88c9f-7c4f-4cd8-b6a1-88b36d543a56 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.324008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb88c9f-7c4f-4cd8-b6a1-88b36d543a56 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.325079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 450/899] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:31.328261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb88c9f-7c4f-4cd8-b6a1-88b36d543a56 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.338601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38049016-a473-41b6-9d09-468982f152d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.339211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38049016-a473-41b6-9d09-468982f152d5 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.339487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38049016-a473-41b6-9d09-468982f152d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.339760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38049016-a473-41b6-9d09-468982f152d5 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.340493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38049016-a473-41b6-9d09-468982f152d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:31.340796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38049016-a473-41b6-9d09-468982f152d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.342282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38049016-a473-41b6-9d09-468982f152d5 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.364063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 450/900] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Sep 1 13:42:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:31.364063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93862a70-6aa7-46c9-a168-48f1b59aae1c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.364063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93862a70-6aa7-46c9-a168-48f1b59aae1c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.364063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93862a70-6aa7-46c9-a168-48f1b59aae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.381042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-93862a70-6aa7-46c9-a168-48f1b59aae1c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:31.382199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-93862a70-6aa7-46c9-a168-48f1b59aae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:31.448561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb88c9f-7c4f-4cd8-b6a1-88b36d543a56 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.452742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 451/901] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:31.452742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93862a70-6aa7-46c9-a168-48f1b59aae1c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.469108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.503210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.503830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.504078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.504346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.505128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:31.505441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.507635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.517463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 451/902] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:31.519714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.523521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.569672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93862a70-6aa7-46c9-a168-48f1b59aae1c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.614444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.615480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.616215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.616646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.632013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5aa6-659b-4b00-a2e6-56b98d8af04e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.673100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.674457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.675190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.675632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.676446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:31.677130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.679536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.691834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 452/903] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:31.694480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.707967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.710384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.729134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.771841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.813730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caadbd61-7ac3-4af7-bd18-f67c010c53f2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.825030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 452/904] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:31 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:42:31.827505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.853750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.854379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.854658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.854940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.855643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] RBAC: Adding query filter params (interface=public, service_id=7928a8e0d99049f1b918e27d252f92cf, region_id=RegionOne) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:31.855947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.857316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:31.862700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 453/905] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:42:31 2022] GET /identity/v3/endpoints?service_id=7928a8e0d99049f1b918e27d252f92cf&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:31.863483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:31.943928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727dbe-7b87-4770-b3a0-f3eb862fa6a8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.958405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:31.974735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27977d2-3b05-46a7-baca-6dd6661ab63f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:31.983864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:31.984840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:31.985129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:31.985407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:31.986415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:31.988000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:32.004619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:32.010355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:32.010978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.011307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.011577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.022688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:32.024124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:32.037549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:32.038707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:32.039869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:32.039869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:32.039869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:32.039869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 454/906] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:32.039869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:32.103792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 453/907] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:31 2022] POST /identity/v3/services => generated 241 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:32.105645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:32.164650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd98cd-0903-4880-a794-9cd73d31ebc5 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:32.216059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:32.216976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.217507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.218265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.219153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:32.221010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:32.230012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 455/908] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:32 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:42:32.289993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f599b78-d31d-4330-a27a-0821830ef1f1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:32.335364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:32.335997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.336275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.336648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.337206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:32.337506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:32.339224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:32.352911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 454/909] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:32.360233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:32.409207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:32.471480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e3e5-a1a4-498c-811d-ef3d2980e203 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:32.507281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:32.508268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.508744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.509229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.516433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:32.518143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:32.530647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:32.531579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:32.532445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:32.534913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 456/910] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:32.541879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:32.579048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:32.654192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7508a83-c15d-45fb-ab25-dee436fe0531 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:32.691858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aceb06-4bcc-4755-b405-e70ce8346e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:32.692481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aceb06-4bcc-4755-b405-e70ce8346e86 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.692759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aceb06-4bcc-4755-b405-e70ce8346e86 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.693028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aceb06-4bcc-4755-b405-e70ce8346e86 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.693739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aceb06-4bcc-4755-b405-e70ce8346e86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:32.694067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aceb06-4bcc-4755-b405-e70ce8346e86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:32.695489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aceb06-4bcc-4755-b405-e70ce8346e86 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:32.708541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 455/911] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:32.718048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:32.821238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad932-6494-4e06-bbeb-4521c7aa26cf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:32.823214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:32.857415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58bbd2-8240-4ad4-86f3-e8916e677e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:32.858679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58bbd2-8240-4ad4-86f3-e8916e677e64 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:32.858955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58bbd2-8240-4ad4-86f3-e8916e677e64 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:32.859223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58bbd2-8240-4ad4-86f3-e8916e677e64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:32.859952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58bbd2-8240-4ad4-86f3-e8916e677e64 None admin] RBAC: Adding query filter params (role.id=57d26c5e32224cb69d4a4e9f819b1e81, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=2dbc06c0703142ba98ffeb72280e1e63) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:32.860254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58bbd2-8240-4ad4-86f3-e8916e677e64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:32.861736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58bbd2-8240-4ad4-86f3-e8916e677e64 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:32.867960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 457/912] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:32 2022] GET /identity/v3/role_assignments?user.id=2dbc06c0703142ba98ffeb72280e1e63&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=57d26c5e32224cb69d4a4e9f819b1e81 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:33.169013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fac041-5b88-4aac-99a0-f62a9918b5ed None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.170030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fac041-5b88-4aac-99a0-f62a9918b5ed None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.170542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fac041-5b88-4aac-99a0-f62a9918b5ed None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.172370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 456/913] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:33.178146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a12b-f570-4cbe-a0f7-836ba015db24 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.178722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a12b-f570-4cbe-a0f7-836ba015db24 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.179232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a12b-f570-4cbe-a0f7-836ba015db24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.207306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-24e2a12b-f570-4cbe-a0f7-836ba015db24 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:33.208855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-24e2a12b-f570-4cbe-a0f7-836ba015db24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:33.284761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 458/914] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:33.563071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fac041-5b88-4aac-99a0-f62a9918b5ed None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:33.663676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:33.671318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fac041-5b88-4aac-99a0-f62a9918b5ed None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:33.707352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:33.708135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.708838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.709120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.709718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:33.711345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:33.720678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:33.721706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:33.722742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 457/915] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:33.730082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a12b-f570-4cbe-a0f7-836ba015db24 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:33.830544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:33.837166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a12b-f570-4cbe-a0f7-836ba015db24 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:33.878354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:33.879084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:33.879362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:33.879629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:33.880261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:33.880560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:33.882061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:33.887852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 459/916] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:33.893685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.000070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789adfa1-375e-4bf7-9515-770d5c103df6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.038738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836e34d-0c10-4b67-8ecb-b9c20d2e9f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.039752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836e34d-0c10-4b67-8ecb-b9c20d2e9f8d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.040233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836e34d-0c10-4b67-8ecb-b9c20d2e9f8d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.040872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836e34d-0c10-4b67-8ecb-b9c20d2e9f8d None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.041814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836e34d-0c10-4b67-8ecb-b9c20d2e9f8d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:34.042321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836e34d-0c10-4b67-8ecb-b9c20d2e9f8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.043879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836e34d-0c10-4b67-8ecb-b9c20d2e9f8d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.053883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 458/917] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Sep 1 13:42:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:34.056324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.135666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbdca6-bd34-4d91-b07d-2419456df2a2 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.136084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbdca6-bd34-4d91-b07d-2419456df2a2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.136255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbdca6-bd34-4d91-b07d-2419456df2a2 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.137704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 459/918] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:34.142852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b50599-bfac-4eb0-a4c4-16aa81943a14 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.143239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b50599-bfac-4eb0-a4c4-16aa81943a14 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.143614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b50599-bfac-4eb0-a4c4-16aa81943a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.162764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-56b50599-bfac-4eb0-a4c4-16aa81943a14 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:34.164095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-56b50599-bfac-4eb0-a4c4-16aa81943a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:34.165439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e057420-979c-448c-a7ca-ec8738e869a8 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.182790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.208233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cebf3-5873-43fe-bb29-a047e1da24d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.208993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cebf3-5873-43fe-bb29-a047e1da24d2 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.209375 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cebf3-5873-43fe-bb29-a047e1da24d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.209748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cebf3-5873-43fe-bb29-a047e1da24d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.210551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cebf3-5873-43fe-bb29-a047e1da24d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.212027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cebf3-5873-43fe-bb29-a047e1da24d2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.244662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 460/919] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:34.296814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 460/920] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:34.299623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1a36-d313-43ec-ae88-6b6ebed6fc30 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.299910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1a36-d313-43ec-ae88-6b6ebed6fc30 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.300178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1a36-d313-43ec-ae88-6b6ebed6fc30 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.301703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 461/921] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:34.310736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9724-2cec-4465-9f13-d5ce478507c2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.310736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9724-2cec-4465-9f13-d5ce478507c2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.310736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9724-2cec-4465-9f13-d5ce478507c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.328735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e23e9724-2cec-4465-9f13-d5ce478507c2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:34.329990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e23e9724-2cec-4465-9f13-d5ce478507c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:34.343807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.419430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 461/922] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:34.530703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d1a36-d313-43ec-ae88-6b6ebed6fc30 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.646671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d1a36-d313-43ec-ae88-6b6ebed6fc30 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.683476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.684464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.685010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.685490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.686900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.688478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.692925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e9724-2cec-4465-9f13-d5ce478507c2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:34.699566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:34.700558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:34.701404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:34.703309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 462/923] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:34.708950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.729325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.815127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e9724-2cec-4465-9f13-d5ce478507c2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.828925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01c04e-2db1-4ffb-ba64-02190b2fb260 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:34.862690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.863782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.864372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.864931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.865954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.866636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:34.867595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:34.867964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:34.868249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.868402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:34.869160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:34.869573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:34.871064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:34.877387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 463/924] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:34.879871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:34.880813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 13:42:34.881712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:34.882988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.884685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 462/925] 10.222.0.61 () {56 vars in 1132 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:34.893822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:34.903784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.973118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:34.998785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e18ff-8a0d-4a42-9fdb-c364ac83231a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.019072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ce96-4cf9-45d0-b143-c5e0e3a33ceb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.043235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0200be5-427e-4d40-9f77-a983f4ff6648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.044510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0200be5-427e-4d40-9f77-a983f4ff6648 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.044799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0200be5-427e-4d40-9f77-a983f4ff6648 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.045080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0200be5-427e-4d40-9f77-a983f4ff6648 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.046732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0200be5-427e-4d40-9f77-a983f4ff6648 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:35.047039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0200be5-427e-4d40-9f77-a983f4ff6648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.049116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0200be5-427e-4d40-9f77-a983f4ff6648 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.060462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee1ca0-8613-4441-99d5-d223875c3125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.060683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 464/926] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:35.061131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1ca0-8613-4441-99d5-d223875c3125 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.061414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1ca0-8613-4441-99d5-d223875c3125 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.061689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1ca0-8613-4441-99d5-d223875c3125 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.062044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b7b6c-c08f-40e9-b66e-67022c55966a None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.062333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b7b6c-c08f-40e9-b66e-67022c55966a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.062427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee1ca0-8613-4441-99d5-d223875c3125 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:35.062603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b7b6c-c08f-40e9-b66e-67022c55966a None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.062710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee1ca0-8613-4441-99d5-d223875c3125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.064101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee1ca0-8613-4441-99d5-d223875c3125 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.068816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 465/927] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:35.068816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b7b6c-c08f-40e9-b66e-67022c55966a None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.073273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 463/928] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:35.076060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506eb2-bb16-427a-ac7b-ac73d9c8af0c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.076755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506eb2-bb16-427a-ac7b-ac73d9c8af0c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.077306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506eb2-bb16-427a-ac7b-ac73d9c8af0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.097708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-42506eb2-bb16-427a-ac7b-ac73d9c8af0c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:35.098965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-42506eb2-bb16-427a-ac7b-ac73d9c8af0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:35.179026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:35.183981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 464/929] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:35.187025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506eb2-bb16-427a-ac7b-ac73d9c8af0c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.188723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b7b6c-c08f-40e9-b66e-67022c55966a None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.238711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.240000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.240876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.241363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.242811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] RBAC: Adding query filter params (interface=public, service_id=f98ff8b7473147209802117329918eb9, region_id=RegionOne) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:35.243343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.245656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.253535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 466/930] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:42:35 2022] GET /identity/v3/endpoints?service_id=f98ff8b7473147209802117329918eb9&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:35.259286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.278837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:35.325281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506eb2-bb16-427a-ac7b-ac73d9c8af0c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.365800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.366965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.367717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.368205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.369077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:35.369583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.371349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.377787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 465/931] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:35.384772 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.398008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911bf75-8fa0-48d8-b1e4-b33e5090e63c None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.456935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.457568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.457883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.458149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.458968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.460118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.466766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 467/932] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:35 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:42:35.467934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.490696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:35.553654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53752f1c-93fd-4445-a97d-feb9cd852258 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.558980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:35.598700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae703bb-0e92-4b32-8527-f87dff103fff None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.604227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.605416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.605941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.606444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.607456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.609119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.644187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.644863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.645141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.645414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.658001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:35.662805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:35.663709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:35.664564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 468/933] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Sep 1 13:42:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:35.670007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:35.719921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 466/934] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:35.810375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6667e8e-025a-4dac-bc5a-3c24803ba52e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:35.850729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:35.867531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:35.868170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:35.868436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:35.868892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:35.869596 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:35.869929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:35.871458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:35.882678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 469/935] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Sep 1 13:42:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:35.888734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:36.023906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411992a-5dcc-4cb2-98e9-276da32ea37e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:36.047393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:36.062409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:36.062968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.063242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.063509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.075601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:36.077579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:36.093926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:36.095028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:36.095907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:36.095907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:36.095907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:36.095907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 467/936] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:36.102407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:36.220641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3520d1-a0f4-4987-ae86-54786dc14c48 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:36.256766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a26bf6-64e0-4851-9c52-f4d841be6e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:36.257381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a26bf6-64e0-4851-9c52-f4d841be6e78 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.257654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a26bf6-64e0-4851-9c52-f4d841be6e78 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.257949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a26bf6-64e0-4851-9c52-f4d841be6e78 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.258602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a26bf6-64e0-4851-9c52-f4d841be6e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:36.258900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a26bf6-64e0-4851-9c52-f4d841be6e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:36.260265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a26bf6-64e0-4851-9c52-f4d841be6e78 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:36.275912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 470/937] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:36.282025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:36.307553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e731-1278-4529-b545-28915a612865 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.307898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e731-1278-4529-b545-28915a612865 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.308099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e731-1278-4529-b545-28915a612865 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.309571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 471/938] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:36.316100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf9cd0-ee0f-4fd9-b3c6-b4d74843ae15 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.316605 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf9cd0-ee0f-4fd9-b3c6-b4d74843ae15 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.317236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf9cd0-ee0f-4fd9-b3c6-b4d74843ae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.337007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4fbf9cd0-ee0f-4fd9-b3c6-b4d74843ae15 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:36.338778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4fbf9cd0-ee0f-4fd9-b3c6-b4d74843ae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:36.408862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab1263-4a83-4fa5-bfae-14f85f499653 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:36.410191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:36.424052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 472/939] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:36.451082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:36.452267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.452538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.452857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.461318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:36.462900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:36.475170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:36.476174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:36.477716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 468/940] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:42:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:36.486769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf9cd0-ee0f-4fd9-b3c6-b4d74843ae15 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:36.593930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf9cd0-ee0f-4fd9-b3c6-b4d74843ae15 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:36.630181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:36.630805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.631082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.631352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.632108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:36.632411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:36.633890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:36.655498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 473/941] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Sep 1 13:42:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:36.661038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:36.702581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:36.706402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:36.759823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:36.790783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26257213-406c-4eb3-95e9-3d25d0110a37 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:36.835810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4928e-b9c3-4618-9a97-179dd4a3fbe8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:36.837476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:36.838475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.838996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.839484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.851141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:36.853064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:36.878452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:36.880257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:36.882804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:36.882804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:36.882804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:36.882804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 469/942] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:36.884393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:36.885006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:36.885369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:36.885606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:36.886197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:36.887876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:36.888723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:36.891737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING keystone.server.flask.common [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 13:42:36.946131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.sql.core [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 13:42:36.946131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 13:42:36.946131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [parameters: {'id': 'b7e0dd7368e944d291b7442306dc80cf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 13:42:36.946131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88134) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b7e0dd7368e944d291b7442306dc80cf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b7e0dd7368e944d291b7442306dc80cf. Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:42:36.968179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application result.read() Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 13:42:36.969712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application yield resource Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise self.value Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application session.commit() Sep 01 13:42:36.980167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self.session.flush() Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application compat.raise_( Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 13:42:36.983555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 13:42:36.987454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application result.read() Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:42:36.990707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [parameters: {'id': 'b7e0dd7368e944d291b7442306dc80cf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:36.999584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:37.002830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b7e0dd7368e944d291b7442306dc80cf. Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 474/943] 10.222.0.61 () {60 vars in 1140 bytes} [Thu Sep 1 13:42:36 2022] POST /identity/v3/projects => generated 262 bytes in 264 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:37.005861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.055687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:37.076891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0151df-9930-4817-b78a-a3ae807f533d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.123949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.125114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.125837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.126325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.127676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:37.128208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.132338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:37.156364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 470/944] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:37.165112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.192647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:37.205442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ceb9-7775-47ff-8771-3712576576eb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.267937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.270153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.270806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.271466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.284244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.287699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:37.309115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:37.311184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:37.313179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:37.316930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 475/945] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:37.325840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.372855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f0af9-87fe-4e76-ba7d-7ffbc6ba3665 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.422696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.423389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.423757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.424121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.431867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.433500 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:37.446165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:37.448164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:37.450231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 471/946] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:37.461433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.468386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:37.483265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157523c-5197-4d2c-8d37-a37a767467e9 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.557005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd7847-229d-47d2-903e-e767fcbdd96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.558155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd7847-229d-47d2-903e-e767fcbdd96a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.558616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd7847-229d-47d2-903e-e767fcbdd96a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.559097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd7847-229d-47d2-903e-e767fcbdd96a None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.561402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd7847-229d-47d2-903e-e767fcbdd96a None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:37.561939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd7847-229d-47d2-903e-e767fcbdd96a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.565006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd7847-229d-47d2-903e-e767fcbdd96a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:37.606265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 476/947] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:37.651515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84e9b8-5591-4a0d-af75-12c7c462f02f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.720807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.722228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.722912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.723592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.725119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:37.725890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.728657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:37.750948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f964fa-6d7d-439b-828f-a582b1f5bdd8 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.751400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f964fa-6d7d-439b-828f-a582b1f5bdd8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.751845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f964fa-6d7d-439b-828f-a582b1f5bdd8 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.760543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 472/949] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:37.760803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 477/949] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:37.763055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f092ef-feb2-4a0c-9ccd-ebd169a57580 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.763524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f092ef-feb2-4a0c-9ccd-ebd169a57580 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.764515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f092ef-feb2-4a0c-9ccd-ebd169a57580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.765506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:37.800017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b9f092ef-feb2-4a0c-9ccd-ebd169a57580 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:37.802415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b9f092ef-feb2-4a0c-9ccd-ebd169a57580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:37.820308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:37.858063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:37.906005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 478/950] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:37.906005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99f15-29b9-4cc5-abab-f64701804296 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.906005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99f15-29b9-4cc5-abab-f64701804296 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.906005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99f15-29b9-4cc5-abab-f64701804296 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.906005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 479/951] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:37.912389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54a986-f1e3-40a3-8d99-3f7da5dd5c81 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.912814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54a986-f1e3-40a3-8d99-3f7da5dd5c81 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.913364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54a986-f1e3-40a3-8d99-3f7da5dd5c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.922527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7576e-6be9-48b6-b615-589526cdf055 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:37.943206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9c54a986-f1e3-40a3-8d99-3f7da5dd5c81 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:37.944464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9c54a986-f1e3-40a3-8d99-3f7da5dd5c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:37.985745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:37.986874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:37.987418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:37.987922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/2dbc06c0703142ba98ffeb72280e1e63/roles/57d26c5e32224cb69d4a4e9f819b1e81` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:37.992245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=2dbc06c0703142ba98ffeb72280e1e63, role_id=57d26c5e32224cb69d4a4e9f819b1e81)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:37.995118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.027991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 480/952] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:38.089614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 473/953] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:37 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/2dbc06c0703142ba98ffeb72280e1e63/roles/57d26c5e32224cb69d4a4e9f819b1e81 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:42:38.192648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54a986-f1e3-40a3-8d99-3f7da5dd5c81 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.281461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.329937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54a986-f1e3-40a3-8d99-3f7da5dd5c81 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.396851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.398456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.399271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.400067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.401469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.404331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:38.422386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:38.424466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:38.426195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:38.427783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.427920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 481/954] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:38.433069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.597305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8919d0-bf33-4447-8eb3-9a0fce40c438 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.604685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf4a73-d8c2-449c-b0fe-079445965d42 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.642458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.650812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.651839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.652347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.659881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.659881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:38.659881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.659881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.662523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 482/955] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:38.663786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.664998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.665571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.666138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.666956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.668072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.669848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.676747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:38.681236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:38.682497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:38.683429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:38.695154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 474/956] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:38.698706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.832206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54331a15-4700-4e3a-aef3-c73bf9bb2659 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.880065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6ae3d-a13d-493f-a738-f3319231601f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:38.883187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.884162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.884686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.885168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.886085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:38.886615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.888417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.899266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 475/957] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:38.900498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:38.946578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:38.947771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:38.948336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:38.948832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:38.950044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:38.950599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:38.953836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:38.964238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 483/958] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:38.968267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:38.969612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:39.026230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bf7df-49c5-484c-87e0-552f19fd50fd None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:39.037705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:39.068694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80a59e-28d3-4b07-9402-9b1a361fc079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:39.069662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80a59e-28d3-4b07-9402-9b1a361fc079 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:39.070140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80a59e-28d3-4b07-9402-9b1a361fc079 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:39.070583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80a59e-28d3-4b07-9402-9b1a361fc079 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:39.071394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80a59e-28d3-4b07-9402-9b1a361fc079 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:39.071991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80a59e-28d3-4b07-9402-9b1a361fc079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:39.073684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80a59e-28d3-4b07-9402-9b1a361fc079 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:39.074820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d91456-7e0d-4375-942c-83938aa84c10 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:39.079265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 476/959] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:39.115468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba962894-7129-4078-8f2e-fa6df3f14912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:39.116144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba962894-7129-4078-8f2e-fa6df3f14912 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:39.116516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba962894-7129-4078-8f2e-fa6df3f14912 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:39.116791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba962894-7129-4078-8f2e-fa6df3f14912 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:39.117747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba962894-7129-4078-8f2e-fa6df3f14912 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:39.119297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba962894-7129-4078-8f2e-fa6df3f14912 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:39.214600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 484/960] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:39.711890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbad32b-deaf-484b-aa77-b1a5072f4568 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:39.712933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbad32b-deaf-484b-aa77-b1a5072f4568 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:39.713518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbad32b-deaf-484b-aa77-b1a5072f4568 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:39.715591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 477/961] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:39.722189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41cf4d-018b-4071-9966-45bb5e6f66e2 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:39.723113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41cf4d-018b-4071-9966-45bb5e6f66e2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:39.723614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41cf4d-018b-4071-9966-45bb5e6f66e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:39.743969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ea41cf4d-018b-4071-9966-45bb5e6f66e2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:39.745551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ea41cf4d-018b-4071-9966-45bb5e6f66e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:39.833892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 485/962] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:40.100839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbad32b-deaf-484b-aa77-b1a5072f4568 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:40.231877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbad32b-deaf-484b-aa77-b1a5072f4568 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:40.257870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:40.275158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:40.275796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:40.276077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:40.276360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:40.277049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:40.278564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:40.281293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING keystone.server.flask.common [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 13:42:40.333468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.sql.core [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 13:42:40.333468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 13:42:40.333468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [parameters: {'user_id': '1981b5a1da0241f5b7b355b8a76ecdc3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 13:42:40.333468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result.read() Sep 01 13:42:40.340526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:42:40.341874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application yield resource Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application next(self.gen) Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise self.value Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application session.commit() Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 13:42:40.343387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.session.flush() Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application compat.raise_( Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 13:42:40.346002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 13:42:40.349488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application conn.query(q) Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application result.read() Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 13:42:40.352901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [parameters: {'user_id': '1981b5a1da0241f5b7b355b8a76ecdc3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:40.356322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 13:42:40.359518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 13:42:40.363160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 13:42:40.363160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 13:42:40.363160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 13:42:40.363160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 13:42:40.363160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 13:42:40.363160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:40.363160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 478/963] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Sep 1 13:42:40 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:40.364672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41cf4d-018b-4071-9966-45bb5e6f66e2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:40.448718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:40.510426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41cf4d-018b-4071-9966-45bb5e6f66e2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:40.549711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d4b75-e468-4535-bbfb-7152b009a283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:40.550838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d4b75-e468-4535-bbfb-7152b009a283 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:40.551296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d4b75-e468-4535-bbfb-7152b009a283 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:40.551727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d4b75-e468-4535-bbfb-7152b009a283 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:40.560029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d4b75-e468-4535-bbfb-7152b009a283 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:40.562404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d4b75-e468-4535-bbfb-7152b009a283 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-ff7d4b75-e468-4535-bbfb-7152b009a283 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:40.577119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:40.578306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 13:42:40.579413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:40.581904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 486/964] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:40.593990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:40.800923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e0e9-bf50-4361-b331-b94acdc1c255 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:40.863609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217b9a-2145-43d7-9e4b-e6c0d74d9443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:40.864647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217b9a-2145-43d7-9e4b-e6c0d74d9443 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:40.865131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217b9a-2145-43d7-9e4b-e6c0d74d9443 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:40.865602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217b9a-2145-43d7-9e4b-e6c0d74d9443 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:40.867226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217b9a-2145-43d7-9e4b-e6c0d74d9443 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:40.867766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217b9a-2145-43d7-9e4b-e6c0d74d9443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:40.870978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217b9a-2145-43d7-9e4b-e6c0d74d9443 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:40.947150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 479/965] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Sep 1 13:42:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:40.950710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edddc-ac42-40cd-96bf-acdd5ba5330d None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:40.951285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edddc-ac42-40cd-96bf-acdd5ba5330d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:40.951601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335edddc-ac42-40cd-96bf-acdd5ba5330d None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:40.954352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 487/966] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:40.966685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231463c6-e20b-499e-b8f7-8da62a7d658a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:40.967653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231463c6-e20b-499e-b8f7-8da62a7d658a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:40.968058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231463c6-e20b-499e-b8f7-8da62a7d658a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:40.999105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-231463c6-e20b-499e-b8f7-8da62a7d658a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:41.001617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-231463c6-e20b-499e-b8f7-8da62a7d658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:41.055451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:41.123455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 480/967] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:41.130986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5bc02-5b39-4858-8455-a7a0bde333f5 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.131327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5bc02-5b39-4858-8455-a7a0bde333f5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.131523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5bc02-5b39-4858-8455-a7a0bde333f5 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.133298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 488/968] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:41.139196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd786-a190-4849-82f0-e0ecda7ddb6b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.139528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd786-a190-4849-82f0-e0ecda7ddb6b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.139751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd786-a190-4849-82f0-e0ecda7ddb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.169037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b24bd786-a190-4849-82f0-e0ecda7ddb6b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:41.171240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b24bd786-a190-4849-82f0-e0ecda7ddb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:41.258976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 481/969] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:41.423416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5bc02-5b39-4858-8455-a7a0bde333f5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:41.528526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5bc02-5b39-4858-8455-a7a0bde333f5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:41.534831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd786-a190-4849-82f0-e0ecda7ddb6b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:41.538665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:41.565138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145194b-10de-43ea-86dd-f631dabcc3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:41.565693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145194b-10de-43ea-86dd-f631dabcc3c9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.566007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145194b-10de-43ea-86dd-f631dabcc3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.566285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145194b-10de-43ea-86dd-f631dabcc3c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.578812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145194b-10de-43ea-86dd-f631dabcc3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-2145194b-10de-43ea-86dd-f631dabcc3c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:41.583774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:41.584991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 489/970] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Sep 1 13:42:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:41.586548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae2e5-34ea-4c4d-9f45-956fc516e347 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.587531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae2e5-34ea-4c4d-9f45-956fc516e347 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.587531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae2e5-34ea-4c4d-9f45-956fc516e347 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.589638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 490/971] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:41.590527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae2e5-34ea-4c4d-9f45-956fc516e347 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:41.638115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:41.644499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd786-a190-4849-82f0-e0ecda7ddb6b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:41.679013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:41.679627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.679900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.680170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.680838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:41.682536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:41.695145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae2e5-34ea-4c4d-9f45-956fc516e347 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:41.728064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ba72b-558f-48a0-8b45-7ef3f1c3225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:41.728666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba72b-558f-48a0-8b45-7ef3f1c3225b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.728942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba72b-558f-48a0-8b45-7ef3f1c3225b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.729211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba72b-558f-48a0-8b45-7ef3f1c3225b None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.729917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ba72b-558f-48a0-8b45-7ef3f1c3225b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:41.730220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ba72b-558f-48a0-8b45-7ef3f1c3225b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:41.731572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ba72b-558f-48a0-8b45-7ef3f1c3225b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:41.739842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 491/972] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Sep 1 13:42:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:41.740961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b072f6-781f-4f5f-9f08-0c43351220ab None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.741239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b072f6-781f-4f5f-9f08-0c43351220ab None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.741508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b072f6-781f-4f5f-9f08-0c43351220ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.759373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-40b072f6-781f-4f5f-9f08-0c43351220ab None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:41.760543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-40b072f6-781f-4f5f-9f08-0c43351220ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:41.773825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 482/973] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:41 2022] POST /identity/v3/services => generated 242 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:41.773825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:41.790640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:41.848524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 492/974] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:41.917085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1feb22-b5aa-42c8-8c28-a09ed6bf9d3c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:41.953738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:41.954479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:41.954783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:41.955060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:41.966329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:41.967818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:41.983435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:41.984565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:41.986285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:41.986285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:41.986285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:41.986285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 483/975] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:41.990920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b072f6-781f-4f5f-9f08-0c43351220ab None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.098686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b072f6-781f-4f5f-9f08-0c43351220ab None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.119108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.132518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.138203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.138844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.139134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.139404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.140059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:42.140364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.141800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.154378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 493/976] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:42.159666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.178786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.231614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d40095-19b1-41b1-a614-c956f5e5d91b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.270161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.270803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.271084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.271362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.271945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.273347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.274388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f971f8-fd7b-44c6-89e0-c4a9047fdc43 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:42.285828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:42.287091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:42.288113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:42.292104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 484/977] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:42.292968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.323986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.324732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.325015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.325280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.333568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.335109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:42.349944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:42.350906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:42.351887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 494/978] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:42.356758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.374242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.407431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2e286-a01e-4a7a-a6b5-479aab70acaa None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.443751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.446707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.447406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.447763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.448098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.448801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:42.449164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.450671 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.460938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 485/979] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:42.462052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.467727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cd278-c47b-46b9-b7b9-4927c1d710df None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.505108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.505712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.506023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.506292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.507034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:42.507319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.508767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.528146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 495/980] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:42.534251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.576943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352f5bc-0832-48df-9e9b-923326ad803e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.622145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.622690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.623041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.623319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.623968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:42.624275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.625659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.628342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.631744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 486/981] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:42.637628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.643866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6ac0a-5b5a-456c-8e2b-d7067624506c None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.683324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.683934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.684212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.684485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.687518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.751939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08e57f-ba06-4515-8c96-01fefb4f710f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.765807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.766942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.789230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.790151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.790432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.790703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.792089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] RBAC: Adding query filter params (interface=admin, service_id=f98ff8b7473147209802117329918eb9, region_id=RegionOne) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:42.792410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.793974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.804721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 487/982] 10.222.0.61 () {56 vars in 1249 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/endpoints?service_id=f98ff8b7473147209802117329918eb9&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:42.805616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.875832 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.881661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 496/983] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 13:42:42.887615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:42.921992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a8251-ff2a-4058-b14b-f2c3f2ef920a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:42.923703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:42.957050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:42.957688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:42.957978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:42.958226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:42.958819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:42.960179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:42.974192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 488/984] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:42:43.000280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fe0d-58e9-4a05-ad33-bbc557b2ca2b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:43.046204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:43.046883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:43.047184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:43.047456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:43.059765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:43.061254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:43.079056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:43.080076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:43.081043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:43.081043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:43.081043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:43.082100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 497/985] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:43.085989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:43.197338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e98215-9bd4-413b-9088-4918413cc22b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:43.224247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:43.233003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:43.233684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:43.237515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:43.237515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:43.237515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:43.237515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:43.237515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:43.249612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 489/986] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:43.255206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:43.353325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:43.371174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df60b3-133d-40f5-a47b-2bcb56ef7f89 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:43.407407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:43.408206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:43.408609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:43.408981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:43.416086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:43.417649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:43.429084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:43.430223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:43.431314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:43.432832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 498/987] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:43.439085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:43.546730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162af8a-53a1-4b7d-bcba-4a4b494796ea None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:43.581994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d873fa-6f52-45d5-acf6-6813bed711ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:43.582760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d873fa-6f52-45d5-acf6-6813bed711ca None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:43.583174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d873fa-6f52-45d5-acf6-6813bed711ca None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:43.583455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d873fa-6f52-45d5-acf6-6813bed711ca None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:43.584156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d873fa-6f52-45d5-acf6-6813bed711ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:43.584504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d873fa-6f52-45d5-acf6-6813bed711ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:43.586107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d873fa-6f52-45d5-acf6-6813bed711ca None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:43.604394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 490/988] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:43.605650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:43.710699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e108a-8981-4a4b-9863-61428d30f0a4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:43.728386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe3b-7c32-4e1d-bc15-309a9bb012aa None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:43.728668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe3b-7c32-4e1d-bc15-309a9bb012aa None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:43.728936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe3b-7c32-4e1d-bc15-309a9bb012aa None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:43.730425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 491/989] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:43.735601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb131a87-aea1-4165-a9b5-d13a1c570bf0 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:43.735885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb131a87-aea1-4165-a9b5-d13a1c570bf0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:43.736155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb131a87-aea1-4165-a9b5-d13a1c570bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:43.745734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:43.746361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:43.746639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:43.746908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:43.747636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] RBAC: Adding query filter params (role.id=57d26c5e32224cb69d4a4e9f819b1e81, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=2dbc06c0703142ba98ffeb72280e1e63) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:43.748006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:43.749938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:43.755410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-bb131a87-aea1-4165-a9b5-d13a1c570bf0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:43.761849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 499/990] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:43 2022] GET /identity/v3/role_assignments?user.id=2dbc06c0703142ba98ffeb72280e1e63&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=57d26c5e32224cb69d4a4e9f819b1e81 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:43.762047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bb131a87-aea1-4165-a9b5-d13a1c570bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:43.815160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:43.846307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 492/991] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:44.112567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:44.125836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:44.223790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154d9ab-2180-4054-a6f1-a080aac795b7 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:44.259381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:44.260009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.260285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.260554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.261134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:44.262543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:44.271874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:44.272874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:44.273811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:44.279058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 500/992] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:44.279795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb131a87-aea1-4165-a9b5-d13a1c570bf0 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:44.395119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb131a87-aea1-4165-a9b5-d13a1c570bf0 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:44.430085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:44.430695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.430976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.431249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.431878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:44.432184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:44.433550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:44.443333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 493/993] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:44.444337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:44.526756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:44.552684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395cc02-e04a-4f72-8414-9e371b6411b4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:44.587371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:44.588032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.588515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.588880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.589533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:44.589872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:44.591252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:44.601543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 501/994] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:44.602818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:44.648890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:44.709049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa176d87-0f6f-4c55-a13b-908af687b6bf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:44.746432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb23c7-5d1b-4202-a0b1-51ccd83aa05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:44.747047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb23c7-5d1b-4202-a0b1-51ccd83aa05a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.747378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb23c7-5d1b-4202-a0b1-51ccd83aa05a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.747626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb23c7-5d1b-4202-a0b1-51ccd83aa05a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.748359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb23c7-5d1b-4202-a0b1-51ccd83aa05a None admin] RBAC: Adding query filter params (interface=public, service_id=92b4886ed5224877be8ef399672aea71, region_id=RegionOne) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:44.748665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb23c7-5d1b-4202-a0b1-51ccd83aa05a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:44.750134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb23c7-5d1b-4202-a0b1-51ccd83aa05a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:44.755815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 494/995] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:42:44 2022] GET /identity/v3/endpoints?service_id=92b4886ed5224877be8ef399672aea71&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:44.761646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:44.872546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a0512-d0ae-45e5-a0e0-60e7f2282703 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:44.909864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17829a86-2232-4cc8-8618-fd0ba226fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:44.910390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17829a86-2232-4cc8-8618-fd0ba226fecd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.910667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17829a86-2232-4cc8-8618-fd0ba226fecd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.910937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17829a86-2232-4cc8-8618-fd0ba226fecd None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.911541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17829a86-2232-4cc8-8618-fd0ba226fecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:44.912896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17829a86-2232-4cc8-8618-fd0ba226fecd None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:44.926048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 502/996] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:44 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:42:44.932505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d91c3-ea2a-4935-a8b1-6e977755f882 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.932796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d91c3-ea2a-4935-a8b1-6e977755f882 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.933074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d91c3-ea2a-4935-a8b1-6e977755f882 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.934513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 495/997] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:44.940536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3377688-b2af-4d67-974a-76cc3352eede None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:44.940823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3377688-b2af-4d67-974a-76cc3352eede None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:44.941103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3377688-b2af-4d67-974a-76cc3352eede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:44.947873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:44.960567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a3377688-b2af-4d67-974a-76cc3352eede None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:44.961804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a3377688-b2af-4d67-974a-76cc3352eede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:45.041998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 503/998] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:45.307245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d91c3-ea2a-4935-a8b1-6e977755f882 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.415239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d91c3-ea2a-4935-a8b1-6e977755f882 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.423541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.451696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.452320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.452594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.452861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.453464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.454865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:45.464020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:45.465258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 13:42:45.466359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:45.469811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 496/999] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:45.475494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3377688-b2af-4d67-974a-76cc3352eede None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.577456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3377688-b2af-4d67-974a-76cc3352eede None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.606572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.611900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27527d1-9ca8-4bc9-948e-8209d1f0e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.612515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27527d1-9ca8-4bc9-948e-8209d1f0e52d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.612790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27527d1-9ca8-4bc9-948e-8209d1f0e52d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.618978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27527d1-9ca8-4bc9-948e-8209d1f0e52d None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.618978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27527d1-9ca8-4bc9-948e-8209d1f0e52d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:45.618978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27527d1-9ca8-4bc9-948e-8209d1f0e52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.618978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27527d1-9ca8-4bc9-948e-8209d1f0e52d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:45.620550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 504/1000] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:45.626561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.713540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1476591-d805-4313-ace9-5fa3446c4f3e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.713910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1476591-d805-4313-ace9-5fa3446c4f3e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.714123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1476591-d805-4313-ace9-5fa3446c4f3e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.715617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 505/1001] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:45.720228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b1c12-ab0e-498f-b622-1433c1fda069 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.720514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b1c12-ab0e-498f-b622-1433c1fda069 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.720790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b1c12-ab0e-498f-b622-1433c1fda069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.735088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506e38-694f-43dd-a4a9-0922d91e9e4b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.738808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0b2b1c12-ab0e-498f-b622-1433c1fda069 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:45.740011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0b2b1c12-ab0e-498f-b622-1433c1fda069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:45.772707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.773315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.773596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.773894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.774523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:45.774825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.776199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:45.786803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 497/1002] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Sep 1 13:42:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:45.788750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:45.816169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 506/1003] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:45.825028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:45.902704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e838d24-2181-4309-a6dd-b909d7b915f4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:45.937954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:45.938567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:45.938844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:45.939114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:45.939780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:45.941154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.029817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 498/1004] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:46.083219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b1c12-ab0e-498f-b622-1433c1fda069 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.194811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b1c12-ab0e-498f-b622-1433c1fda069 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.222019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.237251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.237917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.238201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.238475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] PATH_INFO: `/v3/services/image` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.239055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.240460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:46.249355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:46.250382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:46.251233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 507/1005] 10.222.0.61 () {56 vars in 1105 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:46.257531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.364692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.373003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d32c5-3cc2-4117-92e6-68f51e589be6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.408114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005656b-edb4-4f63-a327-ba819d5cea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.408780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005656b-edb4-4f63-a327-ba819d5cea08 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.410776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005656b-edb4-4f63-a327-ba819d5cea08 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.410776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005656b-edb4-4f63-a327-ba819d5cea08 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.410776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005656b-edb4-4f63-a327-ba819d5cea08 None admin] RBAC: Adding query filter params (name=image) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:46.410776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005656b-edb4-4f63-a327-ba819d5cea08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.411726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005656b-edb4-4f63-a327-ba819d5cea08 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.416632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 499/1006] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.422721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:46.526538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d35c6-ccbc-47ca-abc3-687d8d5c6811 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:46.562415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddede488-d9d4-41f5-8970-8ac32076c97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:46.563030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede488-d9d4-41f5-8970-8ac32076c97a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.563310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede488-d9d4-41f5-8970-8ac32076c97a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.563588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede488-d9d4-41f5-8970-8ac32076c97a None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.564220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddede488-d9d4-41f5-8970-8ac32076c97a None admin] RBAC: Adding query filter params (type=image) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:46.564523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddede488-d9d4-41f5-8970-8ac32076c97a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:46.565918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddede488-d9d4-41f5-8970-8ac32076c97a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:46.577837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 508/1007] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:46.875888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9f996-92b6-438e-8aef-39f54138b832 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.876231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9f996-92b6-438e-8aef-39f54138b832 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.876437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9f996-92b6-438e-8aef-39f54138b832 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.877839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 500/1008] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:46.883008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc26461-61ba-4b35-a8c3-11c4906bbeab None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:46.883297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc26461-61ba-4b35-a8c3-11c4906bbeab None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:46.883595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc26461-61ba-4b35-a8c3-11c4906bbeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:46.904825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-fbc26461-61ba-4b35-a8c3-11c4906bbeab None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:46.906049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fbc26461-61ba-4b35-a8c3-11c4906bbeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:46.953595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:46.982143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 509/1009] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:47.252978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9f996-92b6-438e-8aef-39f54138b832 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.366503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9f996-92b6-438e-8aef-39f54138b832 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.402194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.402882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.403159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.403427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.404059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.405552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:47.414492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:47.415501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:47.416498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 501/1010] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:47.423735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc26461-61ba-4b35-a8c3-11c4906bbeab None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.527008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc26461-61ba-4b35-a8c3-11c4906bbeab None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.565033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.565650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.565981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.566238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.566885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:47.567191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.568559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.578552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 510/1011] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:47.579514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.604675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.686486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8809-8722-465b-8474-00fbda48c509 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.721790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73ab70-cb95-48d6-9e7a-05c27b136ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.722439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73ab70-cb95-48d6-9e7a-05c27b136ef9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.722717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73ab70-cb95-48d6-9e7a-05c27b136ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.722990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73ab70-cb95-48d6-9e7a-05c27b136ef9 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.723614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73ab70-cb95-48d6-9e7a-05c27b136ef9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:47.723914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73ab70-cb95-48d6-9e7a-05c27b136ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.725269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73ab70-cb95-48d6-9e7a-05c27b136ef9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.735279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 502/1012] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Sep 1 13:42:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:47.736777 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:47.785129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:47.838437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde9c43-c0d7-4e7e-b70e-407d714b5b99 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:47.874534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:47.875247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.875416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.875698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.876368 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:47.877732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:47.943956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f82ba-6837-4387-8ff3-d7c4b3a18e96 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.944235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f82ba-6837-4387-8ff3-d7c4b3a18e96 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.944495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f82ba-6837-4387-8ff3-d7c4b3a18e96 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.945871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 503/1013] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:47.950735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a09a8c-aacd-40ef-a2a2-d8318e55fe7e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:47.951042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a09a8c-aacd-40ef-a2a2-d8318e55fe7e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:47.951315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a09a8c-aacd-40ef-a2a2-d8318e55fe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:47.968226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 511/1014] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:47.977047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-89a09a8c-aacd-40ef-a2a2-d8318e55fe7e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:47.977047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-89a09a8c-aacd-40ef-a2a2-d8318e55fe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:48.066122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 504/1015] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:47 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:48.330569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:48.430298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:48.434261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fc24c-e880-437b-99e9-26cd33eed7e4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:48.468359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c98f63-0073-429b-865f-bc1881bd3c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:48.469283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c98f63-0073-429b-865f-bc1881bd3c56 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.469583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c98f63-0073-429b-865f-bc1881bd3c56 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.469967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c98f63-0073-429b-865f-bc1881bd3c56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.471340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c98f63-0073-429b-865f-bc1881bd3c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:48.473276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c98f63-0073-429b-865f-bc1881bd3c56 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:48.477804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 512/1016] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:48.480826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a09a8c-aacd-40ef-a2a2-d8318e55fe7e None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:48.495699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:48.505156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639faf7-1db1-4902-829d-80a353920220 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.505487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639faf7-1db1-4902-829d-80a353920220 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.505715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639faf7-1db1-4902-829d-80a353920220 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.507237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 513/1017] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:48.512053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bc117-9fe4-4c52-8d56-4475a041c9cf None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.512343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bc117-9fe4-4c52-8d56-4475a041c9cf None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.512622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bc117-9fe4-4c52-8d56-4475a041c9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.532306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f78bc117-9fe4-4c52-8d56-4475a041c9cf None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:48.533497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f78bc117-9fe4-4c52-8d56-4475a041c9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:48.597131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a09a8c-aacd-40ef-a2a2-d8318e55fe7e None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:48.606230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 514/1018] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:48 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:48.633890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed2213-0d12-4e5d-84e1-25da94fdeaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:48.634496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed2213-0d12-4e5d-84e1-25da94fdeaba None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:48.634767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed2213-0d12-4e5d-84e1-25da94fdeaba None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:48.635033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed2213-0d12-4e5d-84e1-25da94fdeaba None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:48.635688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed2213-0d12-4e5d-84e1-25da94fdeaba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:48.637223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed2213-0d12-4e5d-84e1-25da94fdeaba None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:48.781796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 505/1019] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:48 2022] POST /identity/v3/projects => generated 311 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:48.881542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bc117-9fe4-4c52-8d56-4475a041c9cf None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:48.988971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:48.993735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bc117-9fe4-4c52-8d56-4475a041c9cf None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:49.030226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb736e-979f-44cd-9c91-9a8c032dc61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:49.030834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb736e-979f-44cd-9c91-9a8c032dc61d None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:49.031109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb736e-979f-44cd-9c91-9a8c032dc61d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:49.031377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb736e-979f-44cd-9c91-9a8c032dc61d None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:49.032022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb736e-979f-44cd-9c91-9a8c032dc61d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:49.033372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb736e-979f-44cd-9c91-9a8c032dc61d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:49.100483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 515/1020] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:42:48 2022] POST /identity/v3/services => generated 242 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:50.039416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77232a-6d6a-4541-9df7-9ae89c5cdff6 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.039960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77232a-6d6a-4541-9df7-9ae89c5cdff6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.039960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77232a-6d6a-4541-9df7-9ae89c5cdff6 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.041448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 506/1021] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:50.046989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305e968-c55e-41d7-9f2a-cb2e32f6bd39 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.047372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305e968-c55e-41d7-9f2a-cb2e32f6bd39 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.047533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305e968-c55e-41d7-9f2a-cb2e32f6bd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.066672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-6305e968-c55e-41d7-9f2a-cb2e32f6bd39 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:50.068019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6305e968-c55e-41d7-9f2a-cb2e32f6bd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:50.143916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 516/1022] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:50.420504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77232a-6d6a-4541-9df7-9ae89c5cdff6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.488819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:50.530280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77232a-6d6a-4541-9df7-9ae89c5cdff6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:50.565034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b14aa2-416f-48b8-b8e4-435f2af89ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:50.566068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b14aa2-416f-48b8-b8e4-435f2af89ef7 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.566542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b14aa2-416f-48b8-b8e4-435f2af89ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.567137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b14aa2-416f-48b8-b8e4-435f2af89ef7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.577979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b14aa2-416f-48b8-b8e4-435f2af89ef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-d4b14aa2-416f-48b8-b8e4-435f2af89ef7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:50.583502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:50.584641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:50.585503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:50.590348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 507/1023] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:50.592561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305e968-c55e-41d7-9f2a-cb2e32f6bd39 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.687847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:50.706746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305e968-c55e-41d7-9f2a-cb2e32f6bd39 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:50.739920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4c1de-9782-4929-b9cb-7ec3eb60f3d9 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.740174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4c1de-9782-4929-b9cb-7ec3eb60f3d9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.740411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4c1de-9782-4929-b9cb-7ec3eb60f3d9 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.745443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 508/1024] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:50.747054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:50.748193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.748469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.748754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.749412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:50.749517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea4761-072f-4ac7-9023-4932bb32eed1 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.749708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:50.749831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea4761-072f-4ac7-9023-4932bb32eed1 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.750027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea4761-072f-4ac7-9023-4932bb32eed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.751107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:50.761213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 517/1025] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:50.767176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:50.770671 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-9cea4761-072f-4ac7-9023-4932bb32eed1 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:50.771869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9cea4761-072f-4ac7-9023-4932bb32eed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:50.855368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 509/1026] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:50.883498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba71628-b3c1-406b-a0ef-978fa7f83a82 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:50.920078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937d9ef-529a-4fdd-8adb-48008455aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:50.920711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937d9ef-529a-4fdd-8adb-48008455aaef None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:50.920988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937d9ef-529a-4fdd-8adb-48008455aaef None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:50.921268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937d9ef-529a-4fdd-8adb-48008455aaef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:50.932459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937d9ef-529a-4fdd-8adb-48008455aaef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:50.933984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937d9ef-529a-4fdd-8adb-48008455aaef None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:50.941456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-8937d9ef-529a-4fdd-8adb-48008455aaef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:50.949857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:50.957121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:50.958356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:50.958356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:50.958356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:50.958356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 518/1027] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:50.958586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea4761-072f-4ac7-9023-4932bb32eed1 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.017180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.046973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb700a-8bca-4f51-b0a8-f25f6199be1b None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.047656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb700a-8bca-4f51-b0a8-f25f6199be1b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.047656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb700a-8bca-4f51-b0a8-f25f6199be1b None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.049381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 519/1028] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:51.054746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc69d7-7fe1-4b98-a3a1-a395d92eff2a None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.055037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc69d7-7fe1-4b98-a3a1-a395d92eff2a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.055314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc69d7-7fe1-4b98-a3a1-a395d92eff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.068051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea4761-072f-4ac7-9023-4932bb32eed1 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.076662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-42cc69d7-7fe1-4b98-a3a1-a395d92eff2a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:51.077881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-42cc69d7-7fe1-4b98-a3a1-a395d92eff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:51.103702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.104580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.105016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.105496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.106318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:51.106762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.109129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.121645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 510/1029] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:51.126355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.150243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 520/1030] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:51.150243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc69d7-7fe1-4b98-a3a1-a395d92eff2a None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.236947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf59bc-e554-44e7-a644-f33ef190c179 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.261093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.265537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc69d7-7fe1-4b98-a3a1-a395d92eff2a None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.289307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.290093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.290485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.290765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.292256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.296951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.296951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 511/1031] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:51.303346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.303593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.312863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.314067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.314067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.314203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.321659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.323182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:51.336097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:51.337478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:51.338840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 521/1032] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:51.343175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.421619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d08c-5567-49d9-8f8f-8725b236159f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.454871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7792-d163-4e92-aece-3b97efbc2096 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.455331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.455938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.456208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.456478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.457111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.458592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.493319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.493949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.494227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.494494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.495253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:51.495548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.496951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.515379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 522/1033] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:51.516170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.562311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.577878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 512/1034] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:42:51 2022] POST /identity/v3/users => generated 285 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:42:51.577878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.625333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0686bc7-2573-46cc-88b8-001b13626a06 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.626143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.666237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61912f29-d21f-4453-bf81-803692476477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.666956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61912f29-d21f-4453-bf81-803692476477 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.667239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61912f29-d21f-4453-bf81-803692476477 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.667517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61912f29-d21f-4453-bf81-803692476477 None admin] PATH_INFO: `/v3/services/image` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.668123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61912f29-d21f-4453-bf81-803692476477 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.669634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61912f29-d21f-4453-bf81-803692476477 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-61912f29-d21f-4453-bf81-803692476477 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:51.679521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:51.680764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:51.681633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 523/1035] 10.222.0.61 () {56 vars in 1105 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:51.689637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61912f29-d21f-4453-bf81-803692476477 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.704896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f2ac6-93ef-42dd-bf89-2a98d034b744 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.744574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.745186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.745459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.745725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.815965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61912f29-d21f-4453-bf81-803692476477 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:51.824311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.826666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.827917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.854593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:51.855184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:51.859566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:51.859566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:51.859566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] RBAC: Adding query filter params (name=image) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:51.859566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:51.859566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:51.863424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 524/1036] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:51.869259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.870123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:51.936597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 513/1037] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:42:51.942159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:51.988043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6c67c-6d91-4007-8bac-3e8330e962c1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.026119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.027091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.027375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.027647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.028584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] RBAC: Adding query filter params (type=image) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.029859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.031415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.042346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 525/1038] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:52.043500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.049640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b9ae-d9fc-42a9-a615-2ab7798f2bdf None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.086192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.086954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.087123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.087373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.090909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.095433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.099192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.100911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:52.115597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:52.116733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:52.118354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:52.118354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:52.118354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:52.118354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 514/1039] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:52.122701 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.159729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1924f28-ffee-4362-8ff6-221371a1dfab None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.201508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.202393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.202671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.202944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] PATH_INFO: `/v3/endpoints` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.203648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] RBAC: Adding query filter params (interface=public, service_id=20a223358e6740bf96534d47e133b1e5, region_id=RegionOne) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.203955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.205440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.210762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 526/1040] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/endpoints?service_id=20a223358e6740bf96534d47e133b1e5&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.216301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.247312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800b415-eb43-4ac0-a470-58a7ba5e8311 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.293416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.294061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.294340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.294607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.295254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.295553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.296917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.319714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 515/1041] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:52.320769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.325333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e052b-ea1d-4f09-a61c-cc783a57defd None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.327072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.361948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.362557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.362831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.363128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.363700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.365051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.373161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 527/1042] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 13:42:52.375076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.431009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5231-cd6d-4482-9d60-8a74e08adf0b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.468749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.469140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.469414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.469685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.477579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.479114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:52.491321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:52.492571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:52.493475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:52.498264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 516/1043] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:52.499624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.608270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dba7c-9a35-414a-8586-d22507b9f31b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.645198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b488e-865d-40ae-bcca-60776f191496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.645830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b488e-865d-40ae-bcca-60776f191496 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.646178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b488e-865d-40ae-bcca-60776f191496 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.646453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b488e-865d-40ae-bcca-60776f191496 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.647156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b488e-865d-40ae-bcca-60776f191496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.647458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b488e-865d-40ae-bcca-60776f191496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.648876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b488e-865d-40ae-bcca-60776f191496 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.665796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 528/1044] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:52.667441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:52.783819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899c07e-d312-48a7-b74a-8b481c0130ad None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:52.794226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:52.819984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217073dc-c7f3-4b7c-ae18-644c66c90e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:52.820811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217073dc-c7f3-4b7c-ae18-644c66c90e0a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:52.821257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217073dc-c7f3-4b7c-ae18-644c66c90e0a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:52.821668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217073dc-c7f3-4b7c-ae18-644c66c90e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:52.822549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217073dc-c7f3-4b7c-ae18-644c66c90e0a None admin] RBAC: Adding query filter params (role.id=57d26c5e32224cb69d4a4e9f819b1e81, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=19a91e958d514c0d8a61289bbe3ea63d) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:52.823014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217073dc-c7f3-4b7c-ae18-644c66c90e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:52.824599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217073dc-c7f3-4b7c-ae18-644c66c90e0a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:52.831345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 517/1045] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:52 2022] GET /identity/v3/role_assignments?user.id=19a91e958d514c0d8a61289bbe3ea63d&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=57d26c5e32224cb69d4a4e9f819b1e81 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:53.547520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588183df-dd35-41f2-84fd-3df994c5ed5e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.548129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588183df-dd35-41f2-84fd-3df994c5ed5e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.548129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588183df-dd35-41f2-84fd-3df994c5ed5e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.549533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 529/1046] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:53.554815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199f74-a856-4516-a1e7-bfbd3b0c5843 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:53.555257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199f74-a856-4516-a1e7-bfbd3b0c5843 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:53.555496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199f74-a856-4516-a1e7-bfbd3b0c5843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:53.574698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-ad199f74-a856-4516-a1e7-bfbd3b0c5843 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:53.576787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ad199f74-a856-4516-a1e7-bfbd3b0c5843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:53.654255 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 518/1047] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:53.922732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588183df-dd35-41f2-84fd-3df994c5ed5e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:53.951677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.029424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588183df-dd35-41f2-84fd-3df994c5ed5e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.064219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.064907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.065188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.065461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.076507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:54.081443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:54.089961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:54.091188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 530/1048] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:54.093366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad199f74-a856-4516-a1e7-bfbd3b0c5843 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.197227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad199f74-a856-4516-a1e7-bfbd3b0c5843 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.199520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.232778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3089e51-046c-4b78-b4b2-d1da1f012735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.233386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089e51-046c-4b78-b4b2-d1da1f012735 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.233673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089e51-046c-4b78-b4b2-d1da1f012735 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.233959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089e51-046c-4b78-b4b2-d1da1f012735 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.234849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3089e51-046c-4b78-b4b2-d1da1f012735 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:54.235321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3089e51-046c-4b78-b4b2-d1da1f012735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:54.237059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3089e51-046c-4b78-b4b2-d1da1f012735 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:54.251074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 519/1049] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:54.253684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.331360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3512b-b84d-440a-858b-8da0c7e006ab None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.331626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3512b-b84d-440a-858b-8da0c7e006ab None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.331883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3512b-b84d-440a-858b-8da0c7e006ab None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.333482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 520/1050] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:54.338112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1e6b-204a-4a9e-bc0a-8418f04fc605 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.338396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1e6b-204a-4a9e-bc0a-8418f04fc605 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.338666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1e6b-204a-4a9e-bc0a-8418f04fc605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.354399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.356399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-81fa1e6b-204a-4a9e-bc0a-8418f04fc605 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:54.358166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-81fa1e6b-204a-4a9e-bc0a-8418f04fc605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:54.370550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac2aef-84e4-4e14-b1f7-292b2a240040 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.407896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.408535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.408821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.409081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] PATH_INFO: `/v3/users/f71991fbe9f34f53a158e50a910d70a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.424634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] RBAC: Authorizing `identity:get_user(user_id=f71991fbe9f34f53a158e50a910d70a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:54.426126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:54.427999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 531/1051] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/users/f71991fbe9f34f53a158e50a910d70a1 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:54.433927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.436410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 521/1052] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:54.535262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d649a2d-b39d-4acb-826b-3acd8ff2eee4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.570286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.570936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.571168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.571834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.586128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.641237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:54.642710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:54.702149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1e6b-204a-4a9e-bc0a-8418f04fc605 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.744724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 532/1053] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 13:42:54.750548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.797111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.800882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:54.806567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1e6b-204a-4a9e-bc0a-8418f04fc605 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.842455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de022b0-bc98-46dd-9373-2dedff83f769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.843090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de022b0-bc98-46dd-9373-2dedff83f769 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.843367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de022b0-bc98-46dd-9373-2dedff83f769 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.843640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de022b0-bc98-46dd-9373-2dedff83f769 None admin] PATH_INFO: `/v3/services/image` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.844225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de022b0-bc98-46dd-9373-2dedff83f769 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:54.845696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de022b0-bc98-46dd-9373-2dedff83f769 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:54.852533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ea64f-0500-48b7-b4ff-aa48d875c639 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-3de022b0-bc98-46dd-9373-2dedff83f769 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:54.854839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:54.855969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:54.857522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 522/1054] 10.222.0.61 () {56 vars in 1105 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:54.858469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6e7fb-1b52-4abf-a482-19add9a12504 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.858469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6e7fb-1b52-4abf-a482-19add9a12504 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.858469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6e7fb-1b52-4abf-a482-19add9a12504 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.860161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 523/1055] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:54.861976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6e7fb-1b52-4abf-a482-19add9a12504 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:54.887514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c7df6-82ce-45aa-b673-3e96622e077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:54.888643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c7df6-82ce-45aa-b673-3e96622e077e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.888919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c7df6-82ce-45aa-b673-3e96622e077e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.889202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c7df6-82ce-45aa-b673-3e96622e077e None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.890353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c7df6-82ce-45aa-b673-3e96622e077e None admin] RBAC: Authorizing `identity:get_project(project_id=7b5f7120487241eb86e8c9f6647a2d23)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:54.891782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c7df6-82ce-45aa-b673-3e96622e077e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:54.893529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 533/1056] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:54.894589 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92731318-0e72-4979-ad81-d385ee06e074 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:54.894866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92731318-0e72-4979-ad81-d385ee06e074 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:54.895135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92731318-0e72-4979-ad81-d385ee06e074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:54.913192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-92731318-0e72-4979-ad81-d385ee06e074 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:54.914418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-92731318-0e72-4979-ad81-d385ee06e074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:54.972710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6e7fb-1b52-4abf-a482-19add9a12504 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:54.981817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 534/1057] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:54.982676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92731318-0e72-4979-ad81-d385ee06e074 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.010749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.011354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.011649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.011923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.012567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] RBAC: Adding query filter params (name=image) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:55.012864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.014762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.020222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 524/1058] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:55.025914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.038982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.055711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.098999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92731318-0e72-4979-ad81-d385ee06e074 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.135899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.137337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.137814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.138807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.139585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99718-02e5-4e7d-ae97-2d9eaaa141a9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.139874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=7b5f7120487241eb86e8c9f6647a2d23, user.id=f71991fbe9f34f53a158e50a910d70a1) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:55.140370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.142975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.149794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 535/1059] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:42:54 2022] GET /identity/v3/role_assignments?user.id=f71991fbe9f34f53a158e50a910d70a1&scope.project.id=7b5f7120487241eb86e8c9f6647a2d23&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:55.176898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.177534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.177852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.178129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.178783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] RBAC: Adding query filter params (type=image) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:55.179085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.180487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.187023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 525/1060] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:55.192619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.253974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.326314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd77cc-4efb-4fd2-a219-8d831fa5ea28 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.335686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.357417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.366617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.367489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.367764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.368051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.368710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.370073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.370258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af2c2f-15a0-41b6-b203-949e70ced60d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.405267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.405955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.406238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.406510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.416377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:55.421515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:55.423822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:55.424845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 526/1061] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:55.432086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.505757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 536/1062] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:42:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:42:55.543940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec70779-89f2-4b1b-b658-613a16bb665f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.593386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.594042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.594413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.594841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.595534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:55.595980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.597232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.609952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 527/1063] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:42:55.613622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.726271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77833151-7688-4f64-b333-68969c6eacbc None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.762459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.763126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.763407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.763681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.774650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.776106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:55.790623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:55.791625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:55.792552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:55.792552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:55.792552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:55.792552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 537/1064] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:55.799307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.808562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:55.905291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43049-14ce-4e03-bd73-2521efafc6e4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:55.939852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:55.940456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:55.940742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:55.941016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:55.941654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:55.941986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:55.943392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:55.955167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 528/1065] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:55.961008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:55.978963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.065629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86871b97-2048-4043-8bd8-ccaf919892b2 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.109475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.110216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.110497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.110770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.118836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.120317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:42:56.132280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:56.133258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:56.134267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 538/1066] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:42:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:56.141633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.211981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.249633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d2e0c-c472-4617-b93e-d3b5168ebcc7 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.292674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.293277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.293606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.293918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.295234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:56.295539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.296939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.319496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 529/1067] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:56.321387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.400368 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.426929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9c80-75cc-4fcb-ba8b-e93a643bdb4c None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.462155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.462712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.462991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.463264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.474860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.476297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:56.491357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:56.492587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:56.493515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:56.493515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:56.493515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:56.493515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 539/1068] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:56.498651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.602995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c776205-40f8-4368-a588-c69ce6053fbb None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.627218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.639979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.640593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.645436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.645436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.645436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:56.645436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.645436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.656258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 530/1069] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:56.661638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.762328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a08a-994d-4107-a95d-60acad28961d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.793939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:56.802974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.807371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.807371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.807371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.811806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.813609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:42:56.824918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:56.826427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:42:56.827345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:56.829584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 540/1070] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:56.833654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:56.939798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc980e2-636f-4e28-bd34-02760165ccc7 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:56.973630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fabb33-d6c5-4dee-b38c-cb9e396bb28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:56.974344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fabb33-d6c5-4dee-b38c-cb9e396bb28c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:56.974686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fabb33-d6c5-4dee-b38c-cb9e396bb28c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:56.975022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fabb33-d6c5-4dee-b38c-cb9e396bb28c None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:56.975779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fabb33-d6c5-4dee-b38c-cb9e396bb28c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:56.976151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fabb33-d6c5-4dee-b38c-cb9e396bb28c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:56.977638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fabb33-d6c5-4dee-b38c-cb9e396bb28c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:56.997721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 531/1071] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:56.999027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.049451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fbf95-c8f4-4b82-bd21-384b7c967245 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.049657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fbf95-c8f4-4b82-bd21-384b7c967245 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.049957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fbf95-c8f4-4b82-bd21-384b7c967245 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.051320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 532/1072] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:57.056275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52971fc-d210-4ee0-baa6-cb73cec62c99 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.056551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52971fc-d210-4ee0-baa6-cb73cec62c99 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.056818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52971fc-d210-4ee0-baa6-cb73cec62c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.075518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e52971fc-d210-4ee0-baa6-cb73cec62c99 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:57.076839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e52971fc-d210-4ee0-baa6-cb73cec62c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:57.116084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddeac3-49b3-4306-81b7-c5159a3a742d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.120324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.148999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 533/1073] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:57.154013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4de5c6-96ae-4960-a1b7-576cbf017c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.154688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4de5c6-96ae-4960-a1b7-576cbf017c15 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.154968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4de5c6-96ae-4960-a1b7-576cbf017c15 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.155243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4de5c6-96ae-4960-a1b7-576cbf017c15 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/19a91e958d514c0d8a61289bbe3ea63d/roles/57d26c5e32224cb69d4a4e9f819b1e81` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.157380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4de5c6-96ae-4960-a1b7-576cbf017c15 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=19a91e958d514c0d8a61289bbe3ea63d, role_id=57d26c5e32224cb69d4a4e9f819b1e81)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:57.158839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4de5c6-96ae-4960-a1b7-576cbf017c15 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:57.235800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 541/1074] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:56 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/19a91e958d514c0d8a61289bbe3ea63d/roles/57d26c5e32224cb69d4a4e9f819b1e81 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:42:57.422878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52971fc-d210-4ee0-baa6-cb73cec62c99 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.510556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8fb65-6db2-463b-9b31-a598d46311cb None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.510866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8fb65-6db2-463b-9b31-a598d46311cb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.511113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8fb65-6db2-463b-9b31-a598d46311cb None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.512596 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 542/1075] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:42:57.517550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e40bd-22ed-431f-b46d-03e67b8032ca None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.517864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e40bd-22ed-431f-b46d-03e67b8032ca None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.518142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e40bd-22ed-431f-b46d-03e67b8032ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.537011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-334e40bd-22ed-431f-b46d-03e67b8032ca None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:57.538235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-334e40bd-22ed-431f-b46d-03e67b8032ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:57.548112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52971fc-d210-4ee0-baa6-cb73cec62c99 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.590159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.607484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.608840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.609312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.610180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.629171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 543/1076] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:42:57.636251 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:57.641431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:57.642446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:42:57.643316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:42:57.647626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 534/1077] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:57.648805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e40bd-22ed-431f-b46d-03e67b8032ca None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.755371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e40bd-22ed-431f-b46d-03e67b8032ca None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.791945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.792848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.793276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.793693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.795102 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:57.795578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:57.797148 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:57.812205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.812847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 544/1078] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:57.850676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.911859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:57.925704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08c1c9-5e90-4756-a4d4-2f87c78331ae None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:57.962379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:57.975198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:57.975840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:57.977885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:57.977885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] PATH_INFO: `/v3/users/f71991fbe9f34f53a158e50a910d70a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:57.979824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] RBAC: Authorizing `identity:get_user(user_id=f71991fbe9f34f53a158e50a910d70a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:57.981314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:57.987634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 535/1079] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/users/f71991fbe9f34f53a158e50a910d70a1 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:57.988857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.049497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b64d8d-7b50-43ad-8074-11bf38993ce8 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.088653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.089266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.089541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.089837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.104282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f8ca-d07d-45e1-94c5-4b19de90c544 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.106004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.140610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.141256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.141528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.141824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.143235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] RBAC: Authorizing `identity:get_project(project_id=7b5f7120487241eb86e8c9f6647a2d23)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.145813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.148112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 536/1080] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:58.154589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.167671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.169151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.213743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.267944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9af3ad-df19-4c3b-88ba-447be5a6214f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.284090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 545/1081] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:42:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 13:42:58.290528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.308375 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.309165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.309520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.310497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23/users/f71991fbe9f34f53a158e50a910d70a1/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.312555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] RBAC: Authorizing `identity:create_grant(project_id=7b5f7120487241eb86e8c9f6647a2d23, user_id=f71991fbe9f34f53a158e50a910d70a1, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.314173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.390317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 537/1082] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:42:58 2022] PUT /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23/users/f71991fbe9f34f53a158e50a910d70a1/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 13:42:58.416768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629ebfd-015b-49ec-8a2e-d367ad2a0ceb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.457298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b510b-5148-4759-ba9d-860f8dc89e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.457966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b510b-5148-4759-ba9d-860f8dc89e4c None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.458248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b510b-5148-4759-ba9d-860f8dc89e4c None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.458573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b510b-5148-4759-ba9d-860f8dc89e4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.466256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.471270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b510b-5148-4759-ba9d-860f8dc89e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.472735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b510b-5148-4759-ba9d-860f8dc89e4c None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-950b510b-5148-4759-ba9d-860f8dc89e4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:42:58.488873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:58.490329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:42:58.491717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:42:58.491717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:42:58.491717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:58.491717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 546/1083] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:42:58.496606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:58.533733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:58.606169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f9cde-262a-4055-9711-79799387e40a None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:58.644105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d89038-ab06-48f9-a37f-c00510fa4f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:58.645093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d89038-ab06-48f9-a37f-c00510fa4f7b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:58.645629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d89038-ab06-48f9-a37f-c00510fa4f7b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:58.646148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d89038-ab06-48f9-a37f-c00510fa4f7b None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:58.647079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d89038-ab06-48f9-a37f-c00510fa4f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:58.647589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d89038-ab06-48f9-a37f-c00510fa4f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:58.649167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d89038-ab06-48f9-a37f-c00510fa4f7b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:58.662031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 538/1084] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:59.158666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92427a-17b4-4cb8-bc9c-da43d31d7b47 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.159296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92427a-17b4-4cb8-bc9c-da43d31d7b47 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.159296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92427a-17b4-4cb8-bc9c-da43d31d7b47 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.160632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 547/1085] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:42:59.165897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d3f5-6356-43e4-909b-0a525e491606 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.166454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d3f5-6356-43e4-909b-0a525e491606 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.166561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d3f5-6356-43e4-909b-0a525e491606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.185054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-92f9d3f5-6356-43e4-909b-0a525e491606 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:42:59.186268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-92f9d3f5-6356-43e4-909b-0a525e491606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:42:59.253403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 539/1086] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:42:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:42:59.529376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92427a-17b4-4cb8-bc9c-da43d31d7b47 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.629574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92427a-17b4-4cb8-bc9c-da43d31d7b47 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:59.663274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:59.663888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.664161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.664427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.674221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:42:59.678852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:42:59.679866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:42:59.680744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 548/1087] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:42:59.686607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9d3f5-6356-43e4-909b-0a525e491606 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.707940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:59.786741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9d3f5-6356-43e4-909b-0a525e491606 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:59.820932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:59.821539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.821849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.822124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.822842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:42:59.823145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:59.824514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:42:59.833419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 540/1088] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:42:59.839424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:42:59.891932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:42:59.943194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc45ef9-1fc6-4b3a-b25a-2680bbbc20e4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:42:59.977126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:42:59.977748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:42:59.978052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:42:59.978327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:42:59.989059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:42:59.990512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:00.005520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:00.006749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:00.007690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:00.007690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:00.007690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:00.007690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 549/1089] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:00.012742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.095680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.113103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99af6c-d634-4085-a294-b365f70d3e52 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.147033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb82a9-4959-42b3-a929-da0c76014745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.147636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb82a9-4959-42b3-a929-da0c76014745 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.147910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb82a9-4959-42b3-a929-da0c76014745 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.148178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb82a9-4959-42b3-a929-da0c76014745 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.148930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb82a9-4959-42b3-a929-da0c76014745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:00.149232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb82a9-4959-42b3-a929-da0c76014745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.150621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb82a9-4959-42b3-a929-da0c76014745 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.162301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 541/1090] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:00.168027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.272764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75783a4e-f457-4329-85ea-bb8eef362049 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.276598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.311084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.311084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.311084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.311084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.320100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.321996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.327222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11110c9-8c10-4de1-8578-88cc260ef167 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.327710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11110c9-8c10-4de1-8578-88cc260ef167 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.327827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11110c9-8c10-4de1-8578-88cc260ef167 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.329215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 542/1091] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:43:00.333997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c6b0-be35-41d3-9b9c-be1e28339984 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.334278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c6b0-be35-41d3-9b9c-be1e28339984 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.334553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c6b0-be35-41d3-9b9c-be1e28339984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:43:00.334859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:00.335912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:00.342314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 550/1092] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:00.344983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.355023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e711c6b0-be35-41d3-9b9c-be1e28339984 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:00.356135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e711c6b0-be35-41d3-9b9c-be1e28339984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:00.423759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 543/1093] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:43:00.446299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0ffc7-8721-491e-a015-5a8a663a6d61 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.480479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.481080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.481368 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.481642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.482404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:00.482703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.484099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.502615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 551/1094] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:00.508467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711c6b0-be35-41d3-9b9c-be1e28339984 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.577892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.606005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711c6b0-be35-41d3-9b9c-be1e28339984 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.639815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.640412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.640685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.640952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.698057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.707953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.709417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.720898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.780445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.799562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3b474-6053-4a5b-a4ba-7a12825ec47e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.812093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 544/1095] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:43:00.817735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.834828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.835476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.835753 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.836021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.846064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:00.850826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:00.851962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:00.853175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 552/1096] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:00.858595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:00.923311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64db32-db71-4d30-a3b8-96c7e1bb0325 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.943939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:00.959245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:00.959884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:00.960158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:00.960430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:00.963243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df4170-c1c2-42c0-b4b9-ffe92ceee67b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:00.970478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:00.971980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:00.986867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:00.987902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:43:00.988818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:00.988818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:43:00.988818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:43:00.988818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 545/1097] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:43:00.993921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.003623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.004235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.004516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.004804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.005490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:01.005816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.007179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.015439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 553/1098] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:01.019182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.021230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.097177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9e5b-b71f-4451-8bd4-9ffe568a7618 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.128923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8ec8-737d-4061-957c-c99976b36ad3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.133839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.134450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.134726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.134996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.135637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:01.135940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.137348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.153024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 546/1099] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:01.154533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.163572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.164203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.164476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.164744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] PATH_INFO: `/v3/users/f71991fbe9f34f53a158e50a910d70a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.165910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] RBAC: Authorizing `identity:get_user(user_id=f71991fbe9f34f53a158e50a910d70a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.167424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.174478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 554/1100] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/users/f71991fbe9f34f53a158e50a910d70a1 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:01.175560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.188587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.239081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.255608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d536487-15df-485a-bb3b-060de2383c93 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.280685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fa9df-5696-47d6-8b1c-87783380f5e0 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.290704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.291339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.291612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.291879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.299059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.300837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:43:01.314833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:01.316017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:43:01.316938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 547/1101] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:01.318136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.318748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.319029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.319308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.321597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.392681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.394065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.405719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.420285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81b9b6-d29e-443a-89cb-2c64bd6eb4d6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.455031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.455637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.455919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.456191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.456887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:01.457262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.458724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.470682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 548/1102] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:01.476846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.480389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.504090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 555/1103] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 13:43:01.509763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.577323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef558e8-e2d0-47ac-bf13-775ffa08c107 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.608365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f829d00-00b9-4032-8dcf-b374bebc30c5 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.612739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.613347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.613621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.613919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.614642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] RBAC: Adding query filter params (role.id=57d26c5e32224cb69d4a4e9f819b1e81, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=19a91e958d514c0d8a61289bbe3ea63d) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:01.614944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.616364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.619742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.637798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 549/1104] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/role_assignments?user.id=19a91e958d514c0d8a61289bbe3ea63d&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=57d26c5e32224cb69d4a4e9f819b1e81 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:01.645416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06117a18-e560-463e-8de6-1cb7eca5a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.646029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06117a18-e560-463e-8de6-1cb7eca5a63b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.646431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06117a18-e560-463e-8de6-1cb7eca5a63b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.646708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06117a18-e560-463e-8de6-1cb7eca5a63b None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.647931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06117a18-e560-463e-8de6-1cb7eca5a63b None admin] RBAC: Authorizing `identity:get_project(project_id=7b5f7120487241eb86e8c9f6647a2d23)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.649436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06117a18-e560-463e-8de6-1cb7eca5a63b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.651345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 556/1105] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:01.658113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:01.736277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:01.772599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46862100-c8b4-4bee-bd0d-e634ab8f87b2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:01.811316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b968fe-d3fc-4f34-b516-9302202a64b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:01.811936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b968fe-d3fc-4f34-b516-9302202a64b3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:01.812215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b968fe-d3fc-4f34-b516-9302202a64b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:01.812489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b968fe-d3fc-4f34-b516-9302202a64b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:01.813224 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b968fe-d3fc-4f34-b516-9302202a64b3 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=7b5f7120487241eb86e8c9f6647a2d23, user.id=f71991fbe9f34f53a158e50a910d70a1) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:01.814301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b968fe-d3fc-4f34-b516-9302202a64b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:01.815024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b968fe-d3fc-4f34-b516-9302202a64b3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:01.821001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 550/1106] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:01 2022] GET /identity/v3/role_assignments?user.id=f71991fbe9f34f53a158e50a910d70a1&scope.project.id=7b5f7120487241eb86e8c9f6647a2d23&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:03.773945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f032a-5c43-4e3a-b1aa-bd7587921717 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.774480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f032a-5c43-4e3a-b1aa-bd7587921717 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.774480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f032a-5c43-4e3a-b1aa-bd7587921717 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.775835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 557/1107] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:03.781750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13de642-0283-47f3-8fb0-db62aa331e5f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:03.782196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13de642-0283-47f3-8fb0-db62aa331e5f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:03.782344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13de642-0283-47f3-8fb0-db62aa331e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:03.800926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f13de642-0283-47f3-8fb0-db62aa331e5f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:03.802150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f13de642-0283-47f3-8fb0-db62aa331e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:03.868948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 551/1108] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:04.138869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f032a-5c43-4e3a-b1aa-bd7587921717 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.306452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f032a-5c43-4e3a-b1aa-bd7587921717 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.340849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a68a58-808e-4445-8470-46dbc91e9019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.341465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68a58-808e-4445-8470-46dbc91e9019 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.341742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68a58-808e-4445-8470-46dbc91e9019 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.342041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68a58-808e-4445-8470-46dbc91e9019 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.343247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a68a58-808e-4445-8470-46dbc91e9019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:04.344679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a68a58-808e-4445-8470-46dbc91e9019 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:04.346277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 558/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:04.352181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13de642-0283-47f3-8fb0-db62aa331e5f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:04.414460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:04.456126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13de642-0283-47f3-8fb0-db62aa331e5f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:04.489836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421935e5-3dfe-4c0e-b6fd-e1a8c22d906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:04.490431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421935e5-3dfe-4c0e-b6fd-e1a8c22d906c None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:04.490702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421935e5-3dfe-4c0e-b6fd-e1a8c22d906c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:04.490990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421935e5-3dfe-4c0e-b6fd-e1a8c22d906c None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:04.491640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421935e5-3dfe-4c0e-b6fd-e1a8c22d906c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:04.493759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421935e5-3dfe-4c0e-b6fd-e1a8c22d906c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:04.583878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 552/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:04 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:06.555040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d607b-e148-4be3-bf59-0cee3aa1f0d1 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.555565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d607b-e148-4be3-bf59-0cee3aa1f0d1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.555565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d607b-e148-4be3-bf59-0cee3aa1f0d1 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.556909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 559/1111] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:06.563348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31f56f-4934-4634-9db4-9e5c8e24bb6a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:06.563730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31f56f-4934-4634-9db4-9e5c8e24bb6a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:06.563814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31f56f-4934-4634-9db4-9e5c8e24bb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:06.582240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4b31f56f-4934-4634-9db4-9e5c8e24bb6a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:06.583435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4b31f56f-4934-4634-9db4-9e5c8e24bb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:06.650460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 553/1112] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:06.927923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d607b-e148-4be3-bf59-0cee3aa1f0d1 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:06.976271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:07.029380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d607b-e148-4be3-bf59-0cee3aa1f0d1 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:07.063779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:07.064403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.064681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.064960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] PATH_INFO: `/v3/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.066384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] RBAC: Authorizing `identity:get_role(role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:07.067851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:07.069169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 560/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:06 2022] GET /identity/v3/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:07.075101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31f56f-4934-4634-9db4-9e5c8e24bb6a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:07.123146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:07.172699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31f56f-4934-4634-9db4-9e5c8e24bb6a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:07.204984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:07.205612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.205927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.206204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] PATH_INFO: `/v3/users/7c4ae800a26b4b66a78870b3c4c3acd4` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.220475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] RBAC: Authorizing `identity:get_user(user_id=7c4ae800a26b4b66a78870b3c4c3acd4)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:07.222670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:07.223616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 554/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:07 2022] GET /identity/v3/users/7c4ae800a26b4b66a78870b3c4c3acd4 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:07.229586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:07.327905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2eeea-612b-4307-a566-2be12e79865a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:07.343300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:07.363241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:07.363839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.364117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.364385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.430616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:07.432048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:07.528653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 561/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:43:07.534601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:07.631633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba20ae-d84f-4788-a479-2dea786a55c9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:07.639367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:07.665723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98d3c-e0cc-48d4-b7a4-ccf615ae7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:07.666396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98d3c-e0cc-48d4-b7a4-ccf615ae7589 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.666780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98d3c-e0cc-48d4-b7a4-ccf615ae7589 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.667057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98d3c-e0cc-48d4-b7a4-ccf615ae7589 None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.668168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d98d3c-e0cc-48d4-b7a4-ccf615ae7589 None admin] RBAC: Authorizing `identity:get_project(project_id=7b5f7120487241eb86e8c9f6647a2d23)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:07.669677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d98d3c-e0cc-48d4-b7a4-ccf615ae7589 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:07.671335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 555/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:07 2022] GET /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:07.677484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:07.692292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:07.785515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26009e-cc2d-4f84-a3f3-8aa0e8564c0e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:07.818592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbfa46-01a6-4c42-9f79-fbdfe86af5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:07.819192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbfa46-01a6-4c42-9f79-fbdfe86af5b8 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:07.819472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbfa46-01a6-4c42-9f79-fbdfe86af5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:07.819744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbfa46-01a6-4c42-9f79-fbdfe86af5b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:07.820591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbfa46-01a6-4c42-9f79-fbdfe86af5b8 None admin] RBAC: Adding query filter params (role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=7b5f7120487241eb86e8c9f6647a2d23, user.id=7c4ae800a26b4b66a78870b3c4c3acd4) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:07.820903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbfa46-01a6-4c42-9f79-fbdfe86af5b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:07.822347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbfa46-01a6-4c42-9f79-fbdfe86af5b8 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:07.828182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 562/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:07 2022] GET /identity/v3/role_assignments?user.id=7c4ae800a26b4b66a78870b3c4c3acd4&scope.project.id=7b5f7120487241eb86e8c9f6647a2d23&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:09.715138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d883b6-7914-4781-ab3c-f3cd541d15ee None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.716236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d883b6-7914-4781-ab3c-f3cd541d15ee None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.716236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d883b6-7914-4781-ab3c-f3cd541d15ee None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.717542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 556/1118] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:09.722734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363f44-7212-4678-b7d5-2c9affc8a146 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:09.723120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363f44-7212-4678-b7d5-2c9affc8a146 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:09.723282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363f44-7212-4678-b7d5-2c9affc8a146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:09.742433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-80363f44-7212-4678-b7d5-2c9affc8a146 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:09.743631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-80363f44-7212-4678-b7d5-2c9affc8a146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:09.819250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 563/1119] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:43:10.091905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d883b6-7914-4781-ab3c-f3cd541d15ee None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.192644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d883b6-7914-4781-ab3c-f3cd541d15ee None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.226635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.227373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.227653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.227929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] PATH_INFO: `/v3/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.229332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] RBAC: Authorizing `identity:get_role(role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.230728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.232068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 557/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:10.237950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363f44-7212-4678-b7d5-2c9affc8a146 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.248728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:10.337502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363f44-7212-4678-b7d5-2c9affc8a146 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.372151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.372696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.372971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.373239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] PATH_INFO: `/v3/users/7c4ae800a26b4b66a78870b3c4c3acd4` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.374365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] RBAC: Authorizing `identity:get_user(user_id=7c4ae800a26b4b66a78870b3c4c3acd4)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.375802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.377369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 564/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/users/7c4ae800a26b4b66a78870b3c4c3acd4 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:10.383465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.430269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:10.487198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf857-adfb-4aac-97f5-72f6b4d8794d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.521873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6f881-c1d1-4a5f-b22b-f4c65fe2db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.522497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f881-c1d1-4a5f-b22b-f4c65fe2db6f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.522776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f881-c1d1-4a5f-b22b-f4c65fe2db6f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.523051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f881-c1d1-4a5f-b22b-f4c65fe2db6f None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.524156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6f881-c1d1-4a5f-b22b-f4c65fe2db6f None admin] RBAC: Authorizing `identity:get_project(project_id=7b5f7120487241eb86e8c9f6647a2d23)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.525635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6f881-c1d1-4a5f-b22b-f4c65fe2db6f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.527300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 558/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:10 2022] GET /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:10.533317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:10.607907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:10.630816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086af51-f624-4514-8dae-fbddf599d33f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:10.663351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65c9ba-761b-42cf-a875-6569f24037c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:10.663995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65c9ba-761b-42cf-a875-6569f24037c3 None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:10.664269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65c9ba-761b-42cf-a875-6569f24037c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:10.664539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65c9ba-761b-42cf-a875-6569f24037c3 None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23/users/7c4ae800a26b4b66a78870b3c4c3acd4/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:10.666672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65c9ba-761b-42cf-a875-6569f24037c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b5f7120487241eb86e8c9f6647a2d23, user_id=7c4ae800a26b4b66a78870b3c4c3acd4, role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:10.668122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65c9ba-761b-42cf-a875-6569f24037c3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:10.760096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 565/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:10 2022] PUT /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23/users/7c4ae800a26b4b66a78870b3c4c3acd4/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:12.631585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dce01a-6b3c-4389-82cf-f03c546b8728 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.632113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dce01a-6b3c-4389-82cf-f03c546b8728 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.632201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dce01a-6b3c-4389-82cf-f03c546b8728 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.633528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 559/1124] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:12.638636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315acff-4526-4e42-9f06-6751fa37c6c0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:12.639041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315acff-4526-4e42-9f06-6751fa37c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:12.639181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315acff-4526-4e42-9f06-6751fa37c6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:12.658440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3315acff-4526-4e42-9f06-6751fa37c6c0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:12.659620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3315acff-4526-4e42-9f06-6751fa37c6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:12.725241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 566/1125] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:12.998129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dce01a-6b3c-4389-82cf-f03c546b8728 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.099460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dce01a-6b3c-4389-82cf-f03c546b8728 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.110489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.133118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.133723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.134025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.134291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] PATH_INFO: `/v3/roles/0b3249b0894e447fa27c9821d06fa7d3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.135696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] RBAC: Authorizing `identity:get_role(role_id=0b3249b0894e447fa27c9821d06fa7d3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.137602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.138961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 560/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:12 2022] GET /identity/v3/roles/0b3249b0894e447fa27c9821d06fa7d3 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.144865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315acff-4526-4e42-9f06-6751fa37c6c0 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.208778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.240516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315acff-4526-4e42-9f06-6751fa37c6c0 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.274249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.274815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.275101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.275353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] PATH_INFO: `/v3/users/7c4ae800a26b4b66a78870b3c4c3acd4` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.276471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] RBAC: Authorizing `identity:get_user(user_id=7c4ae800a26b4b66a78870b3c4c3acd4)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.278003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.279631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 567/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/users/7c4ae800a26b4b66a78870b3c4c3acd4 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.285625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.383263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9fc03-8c3d-4d22-989f-3eec7fd49c46 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.419860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.420467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.420745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.421021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.466557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.487213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.488659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.584922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 561/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 13:43:13.590984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.690011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9023b-de00-46b9-a95f-d5033e99114e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.722796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839499e-f30c-468f-aad2-159335f7a52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.723417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839499e-f30c-468f-aad2-159335f7a52d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.723692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839499e-f30c-468f-aad2-159335f7a52d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.723960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839499e-f30c-468f-aad2-159335f7a52d None admin] PATH_INFO: `/v3/projects/7b5f7120487241eb86e8c9f6647a2d23` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.725059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839499e-f30c-468f-aad2-159335f7a52d None admin] RBAC: Authorizing `identity:get_project(project_id=7b5f7120487241eb86e8c9f6647a2d23)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.726495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839499e-f30c-468f-aad2-159335f7a52d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.728110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 568/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/projects/7b5f7120487241eb86e8c9f6647a2d23 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:13.735000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:13.810569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:13.831773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42620c3-e290-423c-9d9f-93327bc3b406 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:13.866380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70800dfb-e656-4f08-b4b5-95cacc010ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:13.867161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70800dfb-e656-4f08-b4b5-95cacc010ff8 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:13.867438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70800dfb-e656-4f08-b4b5-95cacc010ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:13.867738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70800dfb-e656-4f08-b4b5-95cacc010ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:13.868469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70800dfb-e656-4f08-b4b5-95cacc010ff8 None admin] RBAC: Adding query filter params (role.id=0b3249b0894e447fa27c9821d06fa7d3, scope.project.id=7b5f7120487241eb86e8c9f6647a2d23, user.id=7c4ae800a26b4b66a78870b3c4c3acd4) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:13.868778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70800dfb-e656-4f08-b4b5-95cacc010ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:13.870250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70800dfb-e656-4f08-b4b5-95cacc010ff8 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:13.876609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 562/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:13 2022] GET /identity/v3/role_assignments?user.id=7c4ae800a26b4b66a78870b3c4c3acd4&scope.project.id=7b5f7120487241eb86e8c9f6647a2d23&role.id=0b3249b0894e447fa27c9821d06fa7d3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:15.832073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a480d33-0159-432e-9efd-f5c2697daea2 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:15.832600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a480d33-0159-432e-9efd-f5c2697daea2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:15.832600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a480d33-0159-432e-9efd-f5c2697daea2 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:15.833962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 569/1131] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:15.839277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3f7e-dd85-4d5c-acd4-c526fd99519e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:15.839724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3f7e-dd85-4d5c-acd4-c526fd99519e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:15.839812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3f7e-dd85-4d5c-acd4-c526fd99519e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:15.859293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-077b3f7e-dd85-4d5c-acd4-c526fd99519e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:15.869076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-077b3f7e-dd85-4d5c-acd4-c526fd99519e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:15.927758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 563/1132] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:16.191935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a480d33-0159-432e-9efd-f5c2697daea2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.194914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.294381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a480d33-0159-432e-9efd-f5c2697daea2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:16.327512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b926-2541-45a0-a042-08ea8e39dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:16.328134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b926-2541-45a0-a042-08ea8e39dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.328407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b926-2541-45a0-a042-08ea8e39dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.328676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b926-2541-45a0-a042-08ea8e39dbc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.329802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45b926-2541-45a0-a042-08ea8e39dbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:16.331220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45b926-2541-45a0-a042-08ea8e39dbc9 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:16.332776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 570/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:16.338621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b3f7e-dd85-4d5c-acd4-c526fd99519e None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:16.407609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:16.437358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b3f7e-dd85-4d5c-acd4-c526fd99519e None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:16.472283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f3ee-9826-476c-bd77-e5649037155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:16.472816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f3ee-9826-476c-bd77-e5649037155d None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:16.473098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f3ee-9826-476c-bd77-e5649037155d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:16.473406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f3ee-9826-476c-bd77-e5649037155d None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:16.474097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540f3ee-9826-476c-bd77-e5649037155d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:16.475534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540f3ee-9826-476c-bd77-e5649037155d None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:16.590826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 564/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:43:16 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:18.557796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48142e-aae4-4744-aca2-508f36a18272 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:18.558320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48142e-aae4-4744-aca2-508f36a18272 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:18.558410 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48142e-aae4-4744-aca2-508f36a18272 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:18.560164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 571/1135] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:18.565491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7295f-275c-4eb0-95e9-b099456b1e72 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:18.565892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7295f-275c-4eb0-95e9-b099456b1e72 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:18.566016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7295f-275c-4eb0-95e9-b099456b1e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:18.584558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d0f7295f-275c-4eb0-95e9-b099456b1e72 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:18.585753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d0f7295f-275c-4eb0-95e9-b099456b1e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:18.653161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 565/1136] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:18.926584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48142e-aae4-4744-aca2-508f36a18272 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:18.995660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:19.028267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48142e-aae4-4744-aca2-508f36a18272 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:19.063141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2f843-839f-47c3-9e38-86e13044a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:19.063766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2f843-839f-47c3-9e38-86e13044a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.064044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2f843-839f-47c3-9e38-86e13044a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.064316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2f843-839f-47c3-9e38-86e13044a7b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.065454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2f843-839f-47c3-9e38-86e13044a7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:19.066907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2f843-839f-47c3-9e38-86e13044a7b7 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:19.068476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 572/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:43:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:19.074697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7295f-275c-4eb0-95e9-b099456b1e72 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:19.128997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:19.178435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7295f-275c-4eb0-95e9-b099456b1e72 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:19.211983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449c774-6acf-4782-83cf-e12d4530b22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:19.212852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c774-6acf-4782-83cf-e12d4530b22e None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:19.213136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c774-6acf-4782-83cf-e12d4530b22e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:19.213406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c774-6acf-4782-83cf-e12d4530b22e None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:19.214081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449c774-6acf-4782-83cf-e12d4530b22e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:19.216026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449c774-6acf-4782-83cf-e12d4530b22e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:19.326559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 566/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:19 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:21.259500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674749c5-b653-493f-a84b-5d6a2767a001 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:21.260042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674749c5-b653-493f-a84b-5d6a2767a001 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:21.260042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674749c5-b653-493f-a84b-5d6a2767a001 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:21.261390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 573/1139] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:21.267080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89860-345f-4c19-ad2b-9901f2ae8e40 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:21.267860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89860-345f-4c19-ad2b-9901f2ae8e40 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:21.267860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89860-345f-4c19-ad2b-9901f2ae8e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:21.286176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-81d89860-345f-4c19-ad2b-9901f2ae8e40 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:21.287355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-81d89860-345f-4c19-ad2b-9901f2ae8e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:21.352123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 567/1140] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:21.623014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674749c5-b653-493f-a84b-5d6a2767a001 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:21.724293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674749c5-b653-493f-a84b-5d6a2767a001 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:21.758788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:21.759409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:21.759688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:21.759962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:21.767314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:21.771041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:21.775957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:21.786124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:21.787103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 574/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:21.787832 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d89860-345f-4c19-ad2b-9901f2ae8e40 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:21.827652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:21.884936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d89860-345f-4c19-ad2b-9901f2ae8e40 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:21.917153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:21.917750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:21.918056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:21.918325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:21.919524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:21.919823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:21.921186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:21.928988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 568/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:21.935671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:22.035892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bb6c5-06de-470a-8756-9ef99a306b1a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:22.070364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:22.070984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.071264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.071537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] PATH_INFO: `/v3/users/49e3be23be6f48d38ddb45d8271cb3b4` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.176706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] RBAC: Authorizing `identity:get_user(user_id=49e3be23be6f48d38ddb45d8271cb3b4)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:22.178098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:22.179815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 575/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:21 2022] GET /identity/v3/users/49e3be23be6f48d38ddb45d8271cb3b4 => generated 286 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:22.185883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:22.286897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e880c29-c857-42c8-9a57-e892099c33a9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:22.288486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:22.321238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:22.321879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.322160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.322433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.389213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:22.390779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:22.489758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 569/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:43:22.495950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:22.552315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:22.597879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a1d27-83a1-4736-b7fb-24daea82f3a3 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:22.632319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef171b-d261-4d6b-b89e-c40b82032231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:22.632941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef171b-d261-4d6b-b89e-c40b82032231 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.633219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef171b-d261-4d6b-b89e-c40b82032231 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.633491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef171b-d261-4d6b-b89e-c40b82032231 None admin] PATH_INFO: `/v3/projects/4d12489395e24d1282320b7b6ead5f3c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.634628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef171b-d261-4d6b-b89e-c40b82032231 None admin] RBAC: Authorizing `identity:get_project(project_id=4d12489395e24d1282320b7b6ead5f3c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:22.636039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef171b-d261-4d6b-b89e-c40b82032231 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:22.637663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 576/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:22 2022] GET /identity/v3/projects/4d12489395e24d1282320b7b6ead5f3c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:22.643782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:22.646006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:22.740770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73491226-f713-415a-8c8f-04eb3dc644a0 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:22.773714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914e63e-64ce-4803-863f-1b055eb24d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:22.774338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e63e-64ce-4803-863f-1b055eb24d9b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:22.774616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e63e-64ce-4803-863f-1b055eb24d9b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:22.774887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e63e-64ce-4803-863f-1b055eb24d9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:22.775605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914e63e-64ce-4803-863f-1b055eb24d9b None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=4d12489395e24d1282320b7b6ead5f3c, user.id=49e3be23be6f48d38ddb45d8271cb3b4) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:22.775917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914e63e-64ce-4803-863f-1b055eb24d9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:22.777367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914e63e-64ce-4803-863f-1b055eb24d9b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:22.782692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 570/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:22 2022] GET /identity/v3/role_assignments?user.id=49e3be23be6f48d38ddb45d8271cb3b4&scope.project.id=4d12489395e24d1282320b7b6ead5f3c&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:24.676628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6742e-b6c5-4f52-8865-5259763874e0 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.677222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6742e-b6c5-4f52-8865-5259763874e0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.677222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6742e-b6c5-4f52-8865-5259763874e0 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.678550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 577/1147] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:24.683660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60848a9b-c8fe-4e4b-8cfc-241a09de0278 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:24.684084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60848a9b-c8fe-4e4b-8cfc-241a09de0278 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:24.684247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60848a9b-c8fe-4e4b-8cfc-241a09de0278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:24.702148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-60848a9b-c8fe-4e4b-8cfc-241a09de0278 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:24.703331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-60848a9b-c8fe-4e4b-8cfc-241a09de0278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:24.761552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:24.769449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 571/1148] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:25.038886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6742e-b6c5-4f52-8865-5259763874e0 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.140290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6742e-b6c5-4f52-8865-5259763874e0 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.162777 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.173040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.174154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.174154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.174353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.184497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:25.189335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:25.190563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:25.191819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 578/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:25.196635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60848a9b-c8fe-4e4b-8cfc-241a09de0278 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.294185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60848a9b-c8fe-4e4b-8cfc-241a09de0278 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.328007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.328615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.328890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.329155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.329839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:25.330142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.331503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.339247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 572/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.345035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.441547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2342b8-e611-4653-8b0c-dd6fef2ba030 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.474329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.474945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.475218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.475487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] PATH_INFO: `/v3/users/49e3be23be6f48d38ddb45d8271cb3b4` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.476588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=49e3be23be6f48d38ddb45d8271cb3b4)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.478069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.479624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 579/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/users/49e3be23be6f48d38ddb45d8271cb3b4 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.485562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.543780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.583686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65a45c-0bef-431b-b34b-f85150d9b8a9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.616997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b0c68-1839-4b52-ba49-bc4321efcac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.617606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0c68-1839-4b52-ba49-bc4321efcac2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.617903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0c68-1839-4b52-ba49-bc4321efcac2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.618172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0c68-1839-4b52-ba49-bc4321efcac2 None admin] PATH_INFO: `/v3/projects/4d12489395e24d1282320b7b6ead5f3c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.619262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b0c68-1839-4b52-ba49-bc4321efcac2 None admin] RBAC: Authorizing `identity:get_project(project_id=4d12489395e24d1282320b7b6ead5f3c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.620682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b0c68-1839-4b52-ba49-bc4321efcac2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.622856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 573/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:25 2022] GET /identity/v3/projects/4d12489395e24d1282320b7b6ead5f3c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:25.628770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:25.680639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:25.725672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e8cc6-d536-4e8e-a9a4-809bee8e2ea0 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:25.760139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e577ab5-5c38-4e71-860f-53196a54ca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:25.760719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e577ab5-5c38-4e71-860f-53196a54ca4a None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:25.760998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e577ab5-5c38-4e71-860f-53196a54ca4a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:25.761264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e577ab5-5c38-4e71-860f-53196a54ca4a None admin] PATH_INFO: `/v3/projects/4d12489395e24d1282320b7b6ead5f3c/users/49e3be23be6f48d38ddb45d8271cb3b4/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:25.763161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e577ab5-5c38-4e71-860f-53196a54ca4a None admin] RBAC: Authorizing `identity:create_grant(project_id=4d12489395e24d1282320b7b6ead5f3c, user_id=49e3be23be6f48d38ddb45d8271cb3b4, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:25.764568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e577ab5-5c38-4e71-860f-53196a54ca4a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:25.852508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 580/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:25 2022] PUT /identity/v3/projects/4d12489395e24d1282320b7b6ead5f3c/users/49e3be23be6f48d38ddb45d8271cb3b4/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:27.753911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65549a-f14a-4b6c-8b7a-3a930e6bd370 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.754465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65549a-f14a-4b6c-8b7a-3a930e6bd370 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.754465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65549a-f14a-4b6c-8b7a-3a930e6bd370 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.755805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 574/1154] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:27.760940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c65ca-8592-4024-95ca-36845f921cb8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:27.761322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c65ca-8592-4024-95ca-36845f921cb8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:27.761475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c65ca-8592-4024-95ca-36845f921cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:27.780651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0a8c65ca-8592-4024-95ca-36845f921cb8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:27.781886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0a8c65ca-8592-4024-95ca-36845f921cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:27.847692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 581/1155] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:28.117026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65549a-f14a-4b6c-8b7a-3a930e6bd370 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.217882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65549a-f14a-4b6c-8b7a-3a930e6bd370 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:28.244023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:28.253400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:28.256521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.256521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.256521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.264783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:28.269813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:28.270925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:43:28.272159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 575/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:28.277089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c65ca-8592-4024-95ca-36845f921cb8 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.320181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:28.376601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c65ca-8592-4024-95ca-36845f921cb8 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:28.410728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:28.411325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.411608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.411884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.412547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:28.412853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:28.414236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:28.422827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 582/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:28.428785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.529923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72dc2-3d4a-40a8-a071-dda9ea6eb5bd None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:28.563895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:28.564512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.564789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.565062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] PATH_INFO: `/v3/users/49e3be23be6f48d38ddb45d8271cb3b4` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.566195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=49e3be23be6f48d38ddb45d8271cb3b4)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:28.567633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:28.569193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 576/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/users/49e3be23be6f48d38ddb45d8271cb3b4 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:28.575154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.673941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f89f4b-e40a-4890-9b95-d8491ab0fed4 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:28.676151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:28.706603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:28.707196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:28.707520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:28.707989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:28.773172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:28.774622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:28.872831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 583/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:43:28.878550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:28.937047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:28.982063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c9930-2026-4783-a07e-d03484edfbb3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.015108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072f9f1-2503-4942-9f44-3afbcccc78f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.015731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f9f1-2503-4942-9f44-3afbcccc78f6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.016012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f9f1-2503-4942-9f44-3afbcccc78f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.016282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f9f1-2503-4942-9f44-3afbcccc78f6 None admin] PATH_INFO: `/v3/projects/4d12489395e24d1282320b7b6ead5f3c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.017543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072f9f1-2503-4942-9f44-3afbcccc78f6 None admin] RBAC: Authorizing `identity:get_project(project_id=4d12489395e24d1282320b7b6ead5f3c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.019021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072f9f1-2503-4942-9f44-3afbcccc78f6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.020646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 577/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:28 2022] GET /identity/v3/projects/4d12489395e24d1282320b7b6ead5f3c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:29.027012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:29.030707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:29.128069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3e0a-43a7-4f18-a8d5-d8a964e8313b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:29.162103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2534b8-7970-47ee-a447-a998473badce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:29.162699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2534b8-7970-47ee-a447-a998473badce None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:29.162977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2534b8-7970-47ee-a447-a998473badce None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:29.163248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2534b8-7970-47ee-a447-a998473badce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:29.164007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2534b8-7970-47ee-a447-a998473badce None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=4d12489395e24d1282320b7b6ead5f3c, user.id=49e3be23be6f48d38ddb45d8271cb3b4) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:29.164314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2534b8-7970-47ee-a447-a998473badce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:29.165759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2534b8-7970-47ee-a447-a998473badce None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:29.172151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 584/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:29 2022] GET /identity/v3/role_assignments?user.id=49e3be23be6f48d38ddb45d8271cb3b4&scope.project.id=4d12489395e24d1282320b7b6ead5f3c&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:31.103667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c19e4-6db7-49c1-a3f4-d7d598523490 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.104226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c19e4-6db7-49c1-a3f4-d7d598523490 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.104226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c19e4-6db7-49c1-a3f4-d7d598523490 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.105626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 578/1162] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:31.111819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372a540-02f7-40e7-99ca-06f3e27dfbb6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.112278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372a540-02f7-40e7-99ca-06f3e27dfbb6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.112371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372a540-02f7-40e7-99ca-06f3e27dfbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.131121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d372a540-02f7-40e7-99ca-06f3e27dfbb6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:31.132310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d372a540-02f7-40e7-99ca-06f3e27dfbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:31.194882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.201056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 585/1163] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:31.468026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c19e4-6db7-49c1-a3f4-d7d598523490 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.569598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c19e4-6db7-49c1-a3f4-d7d598523490 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.591650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:31.603205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.609274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.609274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.609274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.678611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.680114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.781635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 579/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:43:31.787597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372a540-02f7-40e7-99ca-06f3e27dfbb6 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.886850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372a540-02f7-40e7-99ca-06f3e27dfbb6 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:31.920761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf7b92-5f7e-4708-8191-6b43f533c32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:31.921406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf7b92-5f7e-4708-8191-6b43f533c32f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:31.921679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf7b92-5f7e-4708-8191-6b43f533c32f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:31.921968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf7b92-5f7e-4708-8191-6b43f533c32f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:31.928464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf7b92-5f7e-4708-8191-6b43f533c32f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:31.929910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf7b92-5f7e-4708-8191-6b43f533c32f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-f5bf7b92-5f7e-4708-8191-6b43f533c32f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:31.941073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:31.942313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:31.943307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 586/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:31.948432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:31.971901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:32.044893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd627c-238b-4d88-8b45-b43ba12f8f12 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:32.077476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216364ef-62f6-44f2-ad0c-b4f2209f6e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:32.078102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216364ef-62f6-44f2-ad0c-b4f2209f6e75 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:32.078377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216364ef-62f6-44f2-ad0c-b4f2209f6e75 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:32.078643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216364ef-62f6-44f2-ad0c-b4f2209f6e75 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:32.079278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216364ef-62f6-44f2-ad0c-b4f2209f6e75 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:32.079575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216364ef-62f6-44f2-ad0c-b4f2209f6e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:32.080918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216364ef-62f6-44f2-ad0c-b4f2209f6e75 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:32.088027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 580/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Sep 1 13:43:31 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:34.005670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9e0d-32d7-41bd-9d83-9fb9577a555e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.006254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9e0d-32d7-41bd-9d83-9fb9577a555e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.006254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9e0d-32d7-41bd-9d83-9fb9577a555e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.007555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 587/1167] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:34.012627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85927c82-2437-45cf-8f63-6c54c96648fb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.013022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85927c82-2437-45cf-8f63-6c54c96648fb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.013178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85927c82-2437-45cf-8f63-6c54c96648fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.031205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-85927c82-2437-45cf-8f63-6c54c96648fb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:34.032415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-85927c82-2437-45cf-8f63-6c54c96648fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:34.098372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 581/1168] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:34.366562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c9e0d-32d7-41bd-9d83-9fb9577a555e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:34.426390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:34.469311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c9e0d-32d7-41bd-9d83-9fb9577a555e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:34.502801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffd696-fffc-4968-8ae4-e04d9063959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:34.503402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffd696-fffc-4968-8ae4-e04d9063959a None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:34.503685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffd696-fffc-4968-8ae4-e04d9063959a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:34.503962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffd696-fffc-4968-8ae4-e04d9063959a None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:34.504626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffd696-fffc-4968-8ae4-e04d9063959a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:34.506021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffd696-fffc-4968-8ae4-e04d9063959a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:34.584231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 588/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Thu Sep 1 13:43:34 2022] POST /identity/v3/domains => generated 267 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:36.549086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf10c1-f2a7-4d98-b179-3938ad525304 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.549641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf10c1-f2a7-4d98-b179-3938ad525304 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.549641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf10c1-f2a7-4d98-b179-3938ad525304 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.551015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 582/1170] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:36.556308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0e07e-4eaa-4524-ba83-014ddda3627e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:36.556724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0e07e-4eaa-4524-ba83-014ddda3627e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:36.556846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0e07e-4eaa-4524-ba83-014ddda3627e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:36.574948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-76c0e07e-4eaa-4524-ba83-014ddda3627e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:36.576110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-76c0e07e-4eaa-4524-ba83-014ddda3627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:36.641668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 589/1171] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:36.914625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf10c1-f2a7-4d98-b179-3938ad525304 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:36.934160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:37.018546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf10c1-f2a7-4d98-b179-3938ad525304 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:37.051955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d7aa5-8d17-4ac2-9ec7-633c4ccdacab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:37.052574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7aa5-8d17-4ac2-9ec7-633c4ccdacab None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:37.052852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7aa5-8d17-4ac2-9ec7-633c4ccdacab None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:37.053125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7aa5-8d17-4ac2-9ec7-633c4ccdacab None admin] PATH_INFO: `/v3/domains/05c599ab1d1847deaee961baaa35a742` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:37.063642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d7aa5-8d17-4ac2-9ec7-633c4ccdacab None admin] RBAC: Authorizing `identity:get_domain(domain_id=05c599ab1d1847deaee961baaa35a742)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:37.065098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d7aa5-8d17-4ac2-9ec7-633c4ccdacab None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:37.066722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 583/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Thu Sep 1 13:43:36 2022] GET /identity/v3/domains/05c599ab1d1847deaee961baaa35a742 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:37.072608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0e07e-4eaa-4524-ba83-014ddda3627e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:37.121654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:37.172597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0e07e-4eaa-4524-ba83-014ddda3627e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:37.206147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba6d72-49f9-4133-8638-d92c42908a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:37.206749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba6d72-49f9-4133-8638-d92c42908a29 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:37.207028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba6d72-49f9-4133-8638-d92c42908a29 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:37.207299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba6d72-49f9-4133-8638-d92c42908a29 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:37.207948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba6d72-49f9-4133-8638-d92c42908a29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:37.209360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba6d72-49f9-4133-8638-d92c42908a29 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:37.298737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 590/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:43:37 2022] POST /identity/v3/projects => generated 361 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:39.229567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb2d89-8487-453a-9006-1fa3b11395b6 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:39.230670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb2d89-8487-453a-9006-1fa3b11395b6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:39.230764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb2d89-8487-453a-9006-1fa3b11395b6 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:39.232056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 584/1174] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:39.237313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c075f-c9f9-4906-81e8-457e4f5e2693 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:39.237732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c075f-c9f9-4906-81e8-457e4f5e2693 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:39.237868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c075f-c9f9-4906-81e8-457e4f5e2693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:39.256542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b87c075f-c9f9-4906-81e8-457e4f5e2693 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:39.257722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b87c075f-c9f9-4906-81e8-457e4f5e2693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:39.324724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 591/1175] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:39.599839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb2d89-8487-453a-9006-1fa3b11395b6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:39.686089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:39.709387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb2d89-8487-453a-9006-1fa3b11395b6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:39.744243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019f045-1753-457d-943f-6f50306a0852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:39.744868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019f045-1753-457d-943f-6f50306a0852 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:39.745146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019f045-1753-457d-943f-6f50306a0852 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:39.745418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019f045-1753-457d-943f-6f50306a0852 None admin] PATH_INFO: `/v3/domains/05c599ab1d1847deaee961baaa35a742` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:39.746548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019f045-1753-457d-943f-6f50306a0852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=05c599ab1d1847deaee961baaa35a742)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:39.748084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019f045-1753-457d-943f-6f50306a0852 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:39.749654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 585/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Thu Sep 1 13:43:39 2022] GET /identity/v3/domains/05c599ab1d1847deaee961baaa35a742 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:39.755473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c075f-c9f9-4906-81e8-457e4f5e2693 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:39.785535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:39.855072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c075f-c9f9-4906-81e8-457e4f5e2693 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:39.889319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74a9e9-e831-4dd3-9881-bf742b31c77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:39.889977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a9e9-e831-4dd3-9881-bf742b31c77d None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:39.890256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a9e9-e831-4dd3-9881-bf742b31c77d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:39.890528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74a9e9-e831-4dd3-9881-bf742b31c77d None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:39.891186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74a9e9-e831-4dd3-9881-bf742b31c77d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:39.892633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74a9e9-e831-4dd3-9881-bf742b31c77d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:40.011910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 592/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:43:39 2022] POST /identity/v3/users => generated 311 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:43:41.986656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2464-8e50-44aa-86a6-a322a3f616d0 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.987216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2464-8e50-44aa-86a6-a322a3f616d0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.987216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2464-8e50-44aa-86a6-a322a3f616d0 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:41.988543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 586/1178] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:41.994151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc5ed5-94ac-45e2-bc8c-a6667987b721 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:41.994593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc5ed5-94ac-45e2-bc8c-a6667987b721 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:41.994690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc5ed5-94ac-45e2-bc8c-a6667987b721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:42.014094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b6bc5ed5-94ac-45e2-bc8c-a6667987b721 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:42.015284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b6bc5ed5-94ac-45e2-bc8c-a6667987b721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:42.080851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 593/1179] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:42.352308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff2464-8e50-44aa-86a6-a322a3f616d0 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:42.453040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff2464-8e50-44aa-86a6-a322a3f616d0 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:42.485518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:42.486173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:42.486455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:42.486729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:42.497228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:42.502000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:42.503081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:43:42.504266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 587/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:42.509209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc5ed5-94ac-45e2-bc8c-a6667987b721 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:42.527291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:42.606771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc5ed5-94ac-45e2-bc8c-a6667987b721 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:42.639923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:42.640519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:42.640794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:42.641063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:42.641721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:42.642048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:42.643403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:42.651461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 594/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:43:42.657491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:42.661801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:42.756295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2606a0-1e4e-4134-b99f-7254281f0a12 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:42.789426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:42.790075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:42.790355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:42.790628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] PATH_INFO: `/v3/users/b2871d6b5bea42bc8a7c7ad5697743e9` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:42.803724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] RBAC: Authorizing `identity:get_user(user_id=b2871d6b5bea42bc8a7c7ad5697743e9)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:42.805162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:42.806802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 588/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:42 2022] GET /identity/v3/users/b2871d6b5bea42bc8a7c7ad5697743e9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:42.813382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:42.891389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:42.912068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837beec4-9260-465a-b57a-de8d467a4024 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:42.945444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:42.946096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:42.946458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:42.946941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:43.013776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:43.015236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:43.090599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:43.111832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 595/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:43:43.118141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:43.179915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:43.218594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b0a8-26a7-4807-8ed6-5e79c3ab5702 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:43.252035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c444db-b50d-473f-ba12-629ef1bf926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:43.252662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c444db-b50d-473f-ba12-629ef1bf926c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:43.252943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c444db-b50d-473f-ba12-629ef1bf926c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:43.253221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c444db-b50d-473f-ba12-629ef1bf926c None admin] PATH_INFO: `/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:43.254645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c444db-b50d-473f-ba12-629ef1bf926c None admin] RBAC: Authorizing `identity:get_project(project_id=25b5dd2c3fce478ab289fa51dedf1e60)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:43.259376 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c444db-b50d-473f-ba12-629ef1bf926c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:43.261025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 589/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:43 2022] GET /identity/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:43.267441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:43.365519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaebb67-674b-45cf-85ce-15efe524facb None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:43.399360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112b769-e022-4b29-aa93-04833750040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:43.399959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112b769-e022-4b29-aa93-04833750040f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:43.400240 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112b769-e022-4b29-aa93-04833750040f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:43.400518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112b769-e022-4b29-aa93-04833750040f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:43.401235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112b769-e022-4b29-aa93-04833750040f None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=25b5dd2c3fce478ab289fa51dedf1e60, user.id=b2871d6b5bea42bc8a7c7ad5697743e9) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:43.401541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112b769-e022-4b29-aa93-04833750040f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:43.403504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112b769-e022-4b29-aa93-04833750040f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:43.408790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 596/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:43 2022] GET /identity/v3/role_assignments?user.id=b2871d6b5bea42bc8a7c7ad5697743e9&scope.project.id=25b5dd2c3fce478ab289fa51dedf1e60&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:45.288232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15836c4a-7f1e-4357-b83a-839d781ff7dc None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.288756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15836c4a-7f1e-4357-b83a-839d781ff7dc None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.288756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15836c4a-7f1e-4357-b83a-839d781ff7dc None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.290204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 590/1186] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:45.295591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73a2c-33d2-429c-ba68-e4df37e897f2 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.296015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73a2c-33d2-429c-ba68-e4df37e897f2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.296140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73a2c-33d2-429c-ba68-e4df37e897f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.314338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d8a73a2c-33d2-429c-ba68-e4df37e897f2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:45.315529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d8a73a2c-33d2-429c-ba68-e4df37e897f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:45.351143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.389031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 597/1187] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:45.654708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15836c4a-7f1e-4357-b83a-839d781ff7dc None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.753783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15836c4a-7f1e-4357-b83a-839d781ff7dc None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.781850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:45.787399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.788016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.788287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.788556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.798230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:45.802891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:45.804172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:45.805274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:43:45.809033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 591/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:45.810158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73a2c-33d2-429c-ba68-e4df37e897f2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:45.912481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73a2c-33d2-429c-ba68-e4df37e897f2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:45.945843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:45.946448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:45.946825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:45.947101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:45.947768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:45.948074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:45.950050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:45.958368 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 598/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:45.964845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.062584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f713fb6-2e02-4126-9e2f-ad656e66e986 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.098053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.098671 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.098949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.099221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] PATH_INFO: `/v3/users/b2871d6b5bea42bc8a7c7ad5697743e9` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.100330 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] RBAC: Authorizing `identity:get_user(user_id=b2871d6b5bea42bc8a7c7ad5697743e9)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.101807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.103404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 592/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:45 2022] GET /identity/v3/users/b2871d6b5bea42bc8a7c7ad5697743e9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.109441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.144390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.208765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b04ce-43ad-4b31-8151-d77d08f20c40 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.240834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c181e2a-80e4-4415-aa1e-632b6512e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.241448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c181e2a-80e4-4415-aa1e-632b6512e135 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.242007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c181e2a-80e4-4415-aa1e-632b6512e135 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.242278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c181e2a-80e4-4415-aa1e-632b6512e135 None admin] PATH_INFO: `/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.243386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c181e2a-80e4-4415-aa1e-632b6512e135 None admin] RBAC: Authorizing `identity:get_project(project_id=25b5dd2c3fce478ab289fa51dedf1e60)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.244789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c181e2a-80e4-4415-aa1e-632b6512e135 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.246967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 599/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:46 2022] GET /identity/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:46.253072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:46.311042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:46.352916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021ea5e-cd29-4250-8aef-173922a03725 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:46.387633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a64d29-922d-4ea2-81c6-c4f86434f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:46.388278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64d29-922d-4ea2-81c6-c4f86434f717 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:46.388559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64d29-922d-4ea2-81c6-c4f86434f717 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:46.388836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64d29-922d-4ea2-81c6-c4f86434f717 None admin] PATH_INFO: `/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60/users/b2871d6b5bea42bc8a7c7ad5697743e9/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:46.390750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a64d29-922d-4ea2-81c6-c4f86434f717 None admin] RBAC: Authorizing `identity:create_grant(project_id=25b5dd2c3fce478ab289fa51dedf1e60, user_id=b2871d6b5bea42bc8a7c7ad5697743e9, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:46.392176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a64d29-922d-4ea2-81c6-c4f86434f717 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:46.473922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 593/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:43:46 2022] PUT /identity/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60/users/b2871d6b5bea42bc8a7c7ad5697743e9/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:43:48.383527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcaa0-7c31-4775-b3a5-0a2ea4ec8b4d None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.384036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcaa0-7c31-4775-b3a5-0a2ea4ec8b4d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.384144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcaa0-7c31-4775-b3a5-0a2ea4ec8b4d None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.385397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 600/1193] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:43:48.390737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287c3c2-4a06-4708-bf52-0c93290e2b31 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.391181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287c3c2-4a06-4708-bf52-0c93290e2b31 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.391273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287c3c2-4a06-4708-bf52-0c93290e2b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.410167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-0287c3c2-4a06-4708-bf52-0c93290e2b31 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:43:48.411346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0287c3c2-4a06-4708-bf52-0c93290e2b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:43:48.476851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 594/1194] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:43:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:43:48.747773 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcaa0-7c31-4775-b3a5-0a2ea4ec8b4d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.851425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcaa0-7c31-4775-b3a5-0a2ea4ec8b4d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:48.852985 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:48.886847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:48.887471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:48.887749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:48.888026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:48.898398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:43:48.903635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:43:48.904715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:43:48.905613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 601/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:43:48.910785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287c3c2-4a06-4708-bf52-0c93290e2b31 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:48.958154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.008748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287c3c2-4a06-4708-bf52-0c93290e2b31 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.044106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.044708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.044991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.045267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.045965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:49.046270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.047643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.055596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 595/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:43:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.062169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.160662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e95cb-e66c-4150-91fe-e9ad961dde2f None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.193625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.194271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.194552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.194827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] PATH_INFO: `/v3/users/b2871d6b5bea42bc8a7c7ad5697743e9` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.195953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] RBAC: Authorizing `identity:get_user(user_id=b2871d6b5bea42bc8a7c7ad5697743e9)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.197385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.199046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 602/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/users/b2871d6b5bea42bc8a7c7ad5697743e9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.205101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.307275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7351876-2c58-4102-be25-b2930c18e5a9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.317880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.341118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.341723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.342030 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.342300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.409473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.410955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.508059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 596/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:43:49.514768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.545405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.614172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690058b-284c-4a5c-9e64-8d8868b52400 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.647957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96468d-81fc-4a24-a89a-468723deeedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.648581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96468d-81fc-4a24-a89a-468723deeedf None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.648861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96468d-81fc-4a24-a89a-468723deeedf None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.649134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96468d-81fc-4a24-a89a-468723deeedf None admin] PATH_INFO: `/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.650282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96468d-81fc-4a24-a89a-468723deeedf None admin] RBAC: Authorizing `identity:get_project(project_id=25b5dd2c3fce478ab289fa51dedf1e60)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.652387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96468d-81fc-4a24-a89a-468723deeedf None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.654042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 603/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/projects/25b5dd2c3fce478ab289fa51dedf1e60 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:49.660421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:43:49.671798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:49.756209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2d229-4900-4d27-8077-09dd49fee06c None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:43:49.789591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cd795-10ba-4376-b90e-972d85e0589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:43:49.790219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cd795-10ba-4376-b90e-972d85e0589a None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:43:49.790493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cd795-10ba-4376-b90e-972d85e0589a None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:43:49.790760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cd795-10ba-4376-b90e-972d85e0589a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:43:49.791468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cd795-10ba-4376-b90e-972d85e0589a None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=25b5dd2c3fce478ab289fa51dedf1e60, user.id=b2871d6b5bea42bc8a7c7ad5697743e9) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:43:49.791765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cd795-10ba-4376-b90e-972d85e0589a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:43:49.793267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cd795-10ba-4376-b90e-972d85e0589a None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:43:49.799049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 597/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:43:49 2022] GET /identity/v3/role_assignments?user.id=b2871d6b5bea42bc8a7c7ad5697743e9&scope.project.id=25b5dd2c3fce478ab289fa51dedf1e60&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:43:59.643679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=51 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:43:59.796237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=63 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:05.682503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2e23-e6d8-4195-badb-306895edc38e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:05.683091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2e23-e6d8-4195-badb-306895edc38e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:05.683091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2e23-e6d8-4195-badb-306895edc38e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:05.684368 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 604/1201] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:05.689711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c15b8-27e9-4247-9af2-b158c8de9771 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:05.690167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c15b8-27e9-4247-9af2-b158c8de9771 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:05.690289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c15b8-27e9-4247-9af2-b158c8de9771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:05.709094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e55c15b8-27e9-4247-9af2-b158c8de9771 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:05.710327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e55c15b8-27e9-4247-9af2-b158c8de9771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:05.779075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 598/1202] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:45:06.052682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d2e23-e6d8-4195-badb-306895edc38e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:06.165755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d2e23-e6d8-4195-badb-306895edc38e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:06.202533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:06.203188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:06.203468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:06.203738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:06.214859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:06.216399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:06.232389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:06.237588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:06.238840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:06.238840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:06.238840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:06.238840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 605/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:06.240910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c15b8-27e9-4247-9af2-b158c8de9771 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:06.352760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c15b8-27e9-4247-9af2-b158c8de9771 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:06.384585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:06.390314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429e9d8-5da6-4195-8bce-ce3c38b8641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:06.390927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429e9d8-5da6-4195-8bce-ce3c38b8641b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:06.391203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429e9d8-5da6-4195-8bce-ce3c38b8641b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:06.391470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429e9d8-5da6-4195-8bce-ce3c38b8641b None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:06.392119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429e9d8-5da6-4195-8bce-ce3c38b8641b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:06.392420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429e9d8-5da6-4195-8bce-ce3c38b8641b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:06.393841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429e9d8-5da6-4195-8bce-ce3c38b8641b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:06.406415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 599/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:06.412426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:06.469723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:06.518555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180a886-4d54-4de4-ae0b-526fc1b0d8d2 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:06.561959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812baef-5a31-447d-ac7c-df45efd076e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:06.562615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812baef-5a31-447d-ac7c-df45efd076e2 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:06.562948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812baef-5a31-447d-ac7c-df45efd076e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:06.563173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812baef-5a31-447d-ac7c-df45efd076e2 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:06.563917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812baef-5a31-447d-ac7c-df45efd076e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:06.565366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812baef-5a31-447d-ac7c-df45efd076e2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:06.724224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 606/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Sep 1 13:45:06 2022] POST /identity/v3/users => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:45:08.710218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cd69e-a01d-468d-a852-62816cc887b6 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:08.710775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cd69e-a01d-468d-a852-62816cc887b6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:08.710775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cd69e-a01d-468d-a852-62816cc887b6 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:08.712103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 600/1206] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:08.717894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf471a80-12b6-41a6-8a59-7c4cbba9f27d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:08.718385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf471a80-12b6-41a6-8a59-7c4cbba9f27d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:08.718485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf471a80-12b6-41a6-8a59-7c4cbba9f27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:08.736769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-cf471a80-12b6-41a6-8a59-7c4cbba9f27d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:08.738005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cf471a80-12b6-41a6-8a59-7c4cbba9f27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:08.807034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 607/1207] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:09.076511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cd69e-a01d-468d-a852-62816cc887b6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:09.185651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cd69e-a01d-468d-a852-62816cc887b6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.222050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.222696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.222976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.223273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.234094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:09.238997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:09.240019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:09.240967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 601/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:45:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:09.246788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf471a80-12b6-41a6-8a59-7c4cbba9f27d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:09.289508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:09.358056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf471a80-12b6-41a6-8a59-7c4cbba9f27d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.396429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.397043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.397422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.397718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.398440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:09.398813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:09.400251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:09.409059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 608/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:45:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:09.415314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:09.469124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:09.520834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9a2d4-2e69-4f63-8ac2-1e32aef4c02d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.555062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.555695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.555977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.556249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.569559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:09.572016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:09.587897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:09.589106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:09.590072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:09.590072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:09.590072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:09.590072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 602/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:09.595930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:09.721224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:09.724150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e6a1b-54fd-4942-a5a8-a4b33c980fa6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.760004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.760602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.761010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.761279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.761953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:09.762253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:09.763609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:09.776199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 609/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:09.782721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:09.900101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fb8b2-af6a-46e5-b295-e2165806b61b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:09.901736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:09.935045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:09.935675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:09.935949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:09.936215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:09.943470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:09.945051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:45:09.957756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:09.959200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:09.960785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 603/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:09.964860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:10.070608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e1864-9318-4322-9a54-4272c6c2d512 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:10.106758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:10.107382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:10.107662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:10.107934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:10.108685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:10.108992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:10.110564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:10.130498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 610/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Sep 1 13:45:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:10.136482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:10.243495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ea47-0acc-48a3-99c0-2e10e0fda7f4 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:10.276950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:10.277556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:10.277870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:10.278130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:10.311260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:10.349302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:10.350786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:10.452915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 604/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:10.458940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:10.478763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:10.561415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad5792-dc8e-4d26-a0a7-7bae3ac4eb07 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:10.598945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:10.599635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:10.599857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:10.600122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:10.612219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:10.613726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:10.629249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:10.630972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:10.631870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:10.631870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:10.631870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:10.631870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 611/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:10.636889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:10.698137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:10.736631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fa307-5176-40c0-a73a-2b5c1df7d6f1 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:10.772497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:10.773092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:10.773365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:10.773634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:10.774295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:10.774591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:10.776481 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:10.788281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 605/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:10.794062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:10.867155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:10.899398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5db66-e6f2-492e-9f53-2175a59ac991 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:10.932227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:10.932851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:10.933123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:10.933389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:10.939756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:10.941160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:45:10.952519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:10.953639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:10.960845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 612/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:10.961689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:11.058467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d056b-e6bb-47b5-9b82-1f93a8f55158 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:11.082036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:11.093223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a935d-a245-4bf6-af2f-e79ef8eec62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:11.093853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a935d-a245-4bf6-af2f-e79ef8eec62c None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:11.094130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a935d-a245-4bf6-af2f-e79ef8eec62c None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:11.094396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a935d-a245-4bf6-af2f-e79ef8eec62c None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:11.095081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a935d-a245-4bf6-af2f-e79ef8eec62c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:11.095380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a935d-a245-4bf6-af2f-e79ef8eec62c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:11.096796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a935d-a245-4bf6-af2f-e79ef8eec62c None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:11.108113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 606/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:11.114262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:11.216284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179f122-5231-457c-ba77-f5f3a1ae2ef1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:11.247611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:11.253681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc72ae2-eeb5-47cd-b567-d980fe118bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:11.254316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc72ae2-eeb5-47cd-b567-d980fe118bf5 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:11.254593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc72ae2-eeb5-47cd-b567-d980fe118bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:11.254862 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc72ae2-eeb5-47cd-b567-d980fe118bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:11.255590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc72ae2-eeb5-47cd-b567-d980fe118bf5 None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=f9d9a39675444fd9929465b27fb83cb3) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:11.255894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc72ae2-eeb5-47cd-b567-d980fe118bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:11.257302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc72ae2-eeb5-47cd-b567-d980fe118bf5 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:11.263324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 613/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:11 2022] GET /identity/v3/role_assignments?user.id=f9d9a39675444fd9929465b27fb83cb3&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:13.148954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d362-857a-4841-8b52-a9eca839cc28 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:13.149506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d362-857a-4841-8b52-a9eca839cc28 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:13.149506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d362-857a-4841-8b52-a9eca839cc28 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:13.150897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 607/1220] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:13.156340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71848b-fa65-42e1-92a7-1f79eeb2983b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:13.156724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71848b-fa65-42e1-92a7-1f79eeb2983b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:13.156896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71848b-fa65-42e1-92a7-1f79eeb2983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:13.175026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8a71848b-fa65-42e1-92a7-1f79eeb2983b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:13.176214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8a71848b-fa65-42e1-92a7-1f79eeb2983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:13.248924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 614/1221] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:13.520669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d362-857a-4841-8b52-a9eca839cc28 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:13.621577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d362-857a-4841-8b52-a9eca839cc28 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:13.653996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:13.654615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:13.654890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:13.655162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:13.664939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:13.670188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:13.671286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:13.672460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 608/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:45:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:13.677462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71848b-fa65-42e1-92a7-1f79eeb2983b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:13.781657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71848b-fa65-42e1-92a7-1f79eeb2983b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:13.807768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:13.818217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:13.818879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:13.819154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:13.819423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:13.820093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] RBAC: Adding query filter params (name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:13.820396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:13.821748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:13.830400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 615/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:13.836523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:13.888975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:13.956129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c679a-df69-4902-a51b-c666d88af46d None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:13.996275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:13.997507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:13.997507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:13.997507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.008054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:14.009546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:14.024609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:14.027084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:14.029344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:14.029344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:14.029344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:14.029344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 609/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:14.033741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:14.139252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce9189-b8bc-4549-954c-94c51a15862e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:14.175091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:14.175762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:14.175961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:14.176229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.176869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:14.177171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:14.178594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:14.190953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 616/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:14.197306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:14.296167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:14.302931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798d502-00df-44b8-89e2-bbf68be72128 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:14.342908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:14.343571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:14.343846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:14.344116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.352709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:14.354212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:45:14.366868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:14.368063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:14.369420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 610/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:14.373960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:14.422749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:14.479671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02364-a123-4acd-b9d4-e67bec26be88 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:14.514050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:14.514661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:14.514927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:14.515195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.515922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:14.516225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:14.517624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:14.535825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 617/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Sep 1 13:45:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:14.541650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:14.647061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9305ee-3a9e-4c80-b7c2-27fb9b84bbf3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:14.680470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:14.681105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:14.681932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:14.682204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.693002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:14.694499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:14.709596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:14.711470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:14.712389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:14.712389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:14.712389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:14.712389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 611/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:14.716816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:14.824569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c5dca-9ad4-429c-a8da-b6adabac8caa None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:14.828236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:14.860024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:14.860635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:14.860912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:14.861181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:14.861857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:14.862165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:14.863565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:14.880346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 618/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:14.887852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:14.897910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:14.988467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adb990-cdfa-439e-85ae-29ab49064d95 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:15.022984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:15.023660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:15.023933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:15.024200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:15.031043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:15.032504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:45:15.045023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:15.046158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:15.047261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 612/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:15.052417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:15.163890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b411fd9-a23e-4019-aaaa-39f7a732ffff None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:15.196546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c88b63-7979-4649-b450-5428075f95c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:15.197721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c88b63-7979-4649-b450-5428075f95c3 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:15.198084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c88b63-7979-4649-b450-5428075f95c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:15.198389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c88b63-7979-4649-b450-5428075f95c3 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:15.199115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c88b63-7979-4649-b450-5428075f95c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:15.199450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c88b63-7979-4649-b450-5428075f95c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:15.201068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c88b63-7979-4649-b450-5428075f95c3 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:15.213743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 619/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:45:15.219749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:15.284998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:15.319185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1300d97-7f7a-4e3f-8c4f-3993e413a088 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:15.352083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79ecdc-83ee-46d8-b3fa-9d899691cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:15.352728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79ecdc-83ee-46d8-b3fa-9d899691cb09 None admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:15.353005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79ecdc-83ee-46d8-b3fa-9d899691cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:15.353272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79ecdc-83ee-46d8-b3fa-9d899691cb09 None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/f9d9a39675444fd9929465b27fb83cb3/roles/61fd046fc53c4a57a1ddd3e4f878622c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:15.367345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79ecdc-83ee-46d8-b3fa-9d899691cb09 None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=f9d9a39675444fd9929465b27fb83cb3, role_id=61fd046fc53c4a57a1ddd3e4f878622c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:15.369658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79ecdc-83ee-46d8-b3fa-9d899691cb09 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:15.512327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 613/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:45:15 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/f9d9a39675444fd9929465b27fb83cb3/roles/61fd046fc53c4a57a1ddd3e4f878622c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:45:17.398403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d4bf-67c0-4c7c-b302-2d3a911c3e5f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:17.398975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d4bf-67c0-4c7c-b302-2d3a911c3e5f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:17.398975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d4bf-67c0-4c7c-b302-2d3a911c3e5f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:17.400959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 620/1233] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:17.406449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dac28-5e88-4383-a539-d9b2410f850a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:17.406874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dac28-5e88-4383-a539-d9b2410f850a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:17.406989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dac28-5e88-4383-a539-d9b2410f850a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:17.425237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-793dac28-5e88-4383-a539-d9b2410f850a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:17.428699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-793dac28-5e88-4383-a539-d9b2410f850a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:17.493843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 614/1234] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:17.762679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d4bf-67c0-4c7c-b302-2d3a911c3e5f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:17.787783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:17.871162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d4bf-67c0-4c7c-b302-2d3a911c3e5f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:17.905823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:17.906448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:17.906723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:17.906998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:17.918207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:17.922990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:17.924800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:17.925663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 621/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Sep 1 13:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:17.930320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dac28-5e88-4383-a539-d9b2410f850a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:17.984177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:18.041401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dac28-5e88-4383-a539-d9b2410f850a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:18.076397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:18.077070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:18.077346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:18.077620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:18.078330 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] RBAC: Adding query filter params (name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:18.078636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:18.080003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:18.088783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 615/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Sep 1 13:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:18.094793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:18.182617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:18.198908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1335d6-364d-422b-b324-c43c75ca167a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:18.235195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:18.235816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:18.236095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:18.236366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:18.247729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:18.249331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:18.265309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:18.266480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:18.267384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:18.267384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:18.267384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:18.267384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 622/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:18.272678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:18.376778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d050b-1718-45c4-a8be-15e0b86bdf86 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:18.387102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:18.411505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:18.412136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:18.412368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:18.412807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:18.413268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:18.413565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:18.414979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:18.427160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 616/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:18.432973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:18.534130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc64ef-d252-47dc-ac2d-1dab1a470662 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:18.567843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:18.568386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:18.568669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:18.568981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] PATH_INFO: `/v3/users/placement` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:18.571718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:18.576982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:18.578497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:45:18.590093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:18.591268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:18.592452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 623/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:18.597173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:18.698444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0774eb-7f99-44c2-b36f-e153166f090e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:18.733602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:18.734241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:18.734521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:18.734796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:18.735536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:18.735844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:18.737296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:18.755120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 617/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Sep 1 13:45:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:18.761788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:18.877034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1ca4-99a4-4278-9af9-3080483f7eef None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:18.911741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:18.912371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:18.912653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:18.912934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:18.974729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:18.981529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:18.982998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:19.081684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 624/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:19.087656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:19.115484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:19.188930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7011a-a83b-4874-8e66-acca178f7964 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:19.222527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:19.223147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:19.223424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:19.223693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:19.234617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:19.236100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:19.250867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:19.251909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:19.252800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:19.252800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:19.252800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:19.252800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 618/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:19.258056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:19.351767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:19.358212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a02b5-d5dc-4251-9a48-f8d5b847232d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:19.391812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:19.392419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:19.392698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:19.393010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:19.393634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:19.393966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:19.395318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:19.406695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 625/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:19.412566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:19.491679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:19.512979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822339dc-313c-40b0-bb8e-c244c962ec89 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:19.544794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:19.545401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:19.545672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:19.545966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:19.552267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:19.553698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:45:19.564562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:19.565632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:19.566986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 619/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:45:19.572084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:19.674697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756d06-b488-4a86-b514-a1a98ea6269b None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:19.709349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c59e2a-1076-4ea9-997d-c24de9f2ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:19.709986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c59e2a-1076-4ea9-997d-c24de9f2ed92 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:19.710266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c59e2a-1076-4ea9-997d-c24de9f2ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:19.710536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c59e2a-1076-4ea9-997d-c24de9f2ed92 None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:19.716243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c59e2a-1076-4ea9-997d-c24de9f2ed92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:19.716243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c59e2a-1076-4ea9-997d-c24de9f2ed92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:19.716243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c59e2a-1076-4ea9-997d-c24de9f2ed92 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:19.725994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 626/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:45:19.732350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:19.839106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade3525-8c12-43d0-8bee-63570932af1f None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:19.874164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01dfea-f0ff-4844-8a13-cfda1935d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:19.874777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01dfea-f0ff-4844-8a13-cfda1935d71e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:19.875054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01dfea-f0ff-4844-8a13-cfda1935d71e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:19.875320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01dfea-f0ff-4844-8a13-cfda1935d71e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:19.876046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01dfea-f0ff-4844-8a13-cfda1935d71e None admin] RBAC: Adding query filter params (role.id=61fd046fc53c4a57a1ddd3e4f878622c, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=f9d9a39675444fd9929465b27fb83cb3) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:19.876346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01dfea-f0ff-4844-8a13-cfda1935d71e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:19.877811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01dfea-f0ff-4844-8a13-cfda1935d71e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:19.883797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 620/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:19 2022] GET /identity/v3/role_assignments?user.id=f9d9a39675444fd9929465b27fb83cb3&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=61fd046fc53c4a57a1ddd3e4f878622c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:21.892392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38592fe-bae0-404c-8e3e-5f43537bf43e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:21.893337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38592fe-bae0-404c-8e3e-5f43537bf43e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:21.893337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38592fe-bae0-404c-8e3e-5f43537bf43e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:21.894951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 627/1247] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:21.900392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bbfba-2877-43d5-9009-1f176fcf9a3d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:21.900863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bbfba-2877-43d5-9009-1f176fcf9a3d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:21.900967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bbfba-2877-43d5-9009-1f176fcf9a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:21.907338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:21.920697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4b2bbfba-2877-43d5-9009-1f176fcf9a3d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:21.922000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4b2bbfba-2877-43d5-9009-1f176fcf9a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:21.991580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 621/1248] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:22.272694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38592fe-bae0-404c-8e3e-5f43537bf43e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:22.297126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:22.397103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38592fe-bae0-404c-8e3e-5f43537bf43e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:22.436972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:22.437601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:22.437910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:22.438182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:22.449498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:22.454346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:22.456119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:22.462073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 628/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:22.463397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bbfba-2877-43d5-9009-1f176fcf9a3d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:22.573020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bbfba-2877-43d5-9009-1f176fcf9a3d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:22.602801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:22.607741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:22.612344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:22.612344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:22.612344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:22.612344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:22.612344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:22.612344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:22.619541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 622/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:22.625666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:22.709620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:22.733233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4baf90-0eac-43d6-9478-13689c2f9a19 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:22.767788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:22.768418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:22.768696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:22.768967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:22.780219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:22.781660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:22.796173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:22.802336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:22.803505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:22.803505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:22.803505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:22.803505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 629/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:22.803791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:22.906271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977fc52-137a-41d2-bbe2-bb30a97ab361 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:22.940267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:22.940866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:22.941147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:22.941422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:22.942097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:22.942403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:22.943778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:22.956253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 623/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:22.962292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:23.067375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5a8ea-b7a5-4759-ae5e-506934bfa977 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:23.095856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:23.105699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:23.106532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.106919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.107279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.115612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:23.117316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:45:23.128996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:23.130202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:23.131161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 630/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:23.136967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:23.192965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:23.236733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74e61e-c022-4a09-b3a0-3216046e7043 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:23.271706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:23.272311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.272597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.272867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.273630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:23.273963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:23.275506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:23.294674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 624/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:23.300470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:23.405572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772edd-2797-4d5d-817b-edb9843562c1 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:23.444438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:23.444988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.445265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.445534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.505107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:23.517928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:23.519435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:23.620545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 631/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:23.626662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:23.729129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10513b49-e4d0-427e-9e76-2fcc85e485d3 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:23.742685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:23.763203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:23.763826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.764104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.764370 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.774920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:23.776372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:23.791930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:23.799568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:23.800663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:23.800663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:23.800663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:23.800663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 625/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:23.800947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:23.888893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:23.901083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac7510-e2b3-401f-a87b-f39973006371 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:23.936110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:23.936715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:23.936996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:23.937269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:23.937942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:23.938250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:23.939606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:23.951760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 632/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:23.957747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:24.061871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7cc3-ff83-4ebd-a1e7-1a90509748a1 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:24.103752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:24.104395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:24.104676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:24.104981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:24.112171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:24.113622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:45:24.127018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:24.128104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:24.129205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 626/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:45:24.134175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:24.234800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472b29c-f10a-4b69-96fe-6a5c234c5e78 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:24.269161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d961f16-a3ae-4b37-b2f0-e4889f17ff3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:24.269761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d961f16-a3ae-4b37-b2f0-e4889f17ff3e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:24.270065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d961f16-a3ae-4b37-b2f0-e4889f17ff3e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:24.270333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d961f16-a3ae-4b37-b2f0-e4889f17ff3e None admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:24.271018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d961f16-a3ae-4b37-b2f0-e4889f17ff3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:24.271320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d961f16-a3ae-4b37-b2f0-e4889f17ff3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:24.272714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d961f16-a3ae-4b37-b2f0-e4889f17ff3e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:24.284944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 633/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:24.291566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:24.318056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:24.391729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7e8a-1bf5-4e85-a6e5-44c712440aa2 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:24.426393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0084a-5929-40a4-8048-ce8f1186df17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:24.426997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0084a-5929-40a4-8048-ce8f1186df17 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:24.427270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0084a-5929-40a4-8048-ce8f1186df17 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:24.427537 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0084a-5929-40a4-8048-ce8f1186df17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:24.428266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0084a-5929-40a4-8048-ce8f1186df17 None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=f9d9a39675444fd9929465b27fb83cb3) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:24.428567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0084a-5929-40a4-8048-ce8f1186df17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:24.430107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0084a-5929-40a4-8048-ce8f1186df17 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:24.435635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 627/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:24 2022] GET /identity/v3/role_assignments?user.id=f9d9a39675444fd9929465b27fb83cb3&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:26.365695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3fe21-590b-46d4-bebc-c02ff8a4ce56 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:26.366549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3fe21-590b-46d4-bebc-c02ff8a4ce56 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:26.366650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3fe21-590b-46d4-bebc-c02ff8a4ce56 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:26.367980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 634/1261] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:26.373342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865946a-6830-4166-8121-57a9f393c1e2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:26.373831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865946a-6830-4166-8121-57a9f393c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:26.373942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865946a-6830-4166-8121-57a9f393c1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:26.392587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c865946a-6830-4166-8121-57a9f393c1e2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:26.393812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c865946a-6830-4166-8121-57a9f393c1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:26.466852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 628/1262] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:26.738497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3fe21-590b-46d4-bebc-c02ff8a4ce56 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:26.749726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:26.839951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3fe21-590b-46d4-bebc-c02ff8a4ce56 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:26.880881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:26.881511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:26.881882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:26.882159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:26.892552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:26.897393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:26.898531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:26.900321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 635/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:26.905227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865946a-6830-4166-8121-57a9f393c1e2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:26.924504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.004368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865946a-6830-4166-8121-57a9f393c1e2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.039658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.040268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.040548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.040821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.041486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:27.041808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.043180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.051225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 629/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:27.057147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.125907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.159526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded72c6-a838-44c3-ad6d-c2d1f383c6ed None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.192965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.193592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.193899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.194172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.204950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.206411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:27.220647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:27.221916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:27.223318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:27.223318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:27.223318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:27.223318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 636/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:27.227905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.324591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.352414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3138e-4da7-4b3b-ad11-ae82f5459bb6 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.393952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.394772 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.394916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.395052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.395724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:27.396022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.397407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.409133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 630/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:27.415416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.523568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd769d-f92b-478f-9a38-59a98897a892 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.543647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.558332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.558963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.559243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.559515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.567210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.568595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:45:27.580201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:27.581191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:27.582188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 637/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:45:27.587311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.698678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394619ea-c597-4f63-86cc-ca9c09252a4e None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.734974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.735599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.735878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.736152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.736898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:27.737204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.738694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.750158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:27.759855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 631/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:27.765993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:27.871395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc619e-f983-44c7-bd00-d7aaffce72d6 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:27.904513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:27.905139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:27.905414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:27.905682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:27.916693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:27.918166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:27.932881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:27.937811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:27.938931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:27.938931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:27.938931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:27.938931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 638/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:27.940205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:28.040878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c5d0e-98e5-4ea1-9812-a197b11e7b2b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:28.075495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:28.076101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:28.076380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:28.076652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:28.077293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:28.077595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:28.079010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:28.090935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 632/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:28.096836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:28.131515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:28.209609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ee655-3969-4eb5-a70b-ea2c25c5051a None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:28.244423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:28.245055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:28.245335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:28.245608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:28.252418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:28.253992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:45:28.266047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:28.271681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:28.272890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 639/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:28.273608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:28.331606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:28.374538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf5caf-9735-45dc-823a-b27cc54b2e35 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:28.408139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7326bdc-b79f-4975-aed3-d3f0ed30a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:28.408779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326bdc-b79f-4975-aed3-d3f0ed30a532 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:28.408967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326bdc-b79f-4975-aed3-d3f0ed30a532 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:28.409239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326bdc-b79f-4975-aed3-d3f0ed30a532 None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:28.409956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7326bdc-b79f-4975-aed3-d3f0ed30a532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:28.410262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7326bdc-b79f-4975-aed3-d3f0ed30a532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:28.411696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7326bdc-b79f-4975-aed3-d3f0ed30a532 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:28.424222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 633/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:45:28.430535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:28.524934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:28.533443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b464b8-0949-492f-bdb4-226f86b9863e None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:28.566370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e659e7-1a9a-47d3-b942-4846bc07f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:28.567002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e659e7-1a9a-47d3-b942-4846bc07f4cb None admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:28.567279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e659e7-1a9a-47d3-b942-4846bc07f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:28.567551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e659e7-1a9a-47d3-b942-4846bc07f4cb None admin] PATH_INFO: `/v3/projects/a745287849ba435e8bd1b543d58d9307/users/f9d9a39675444fd9929465b27fb83cb3/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:28.569448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e659e7-1a9a-47d3-b942-4846bc07f4cb None admin] RBAC: Authorizing `identity:create_grant(project_id=a745287849ba435e8bd1b543d58d9307, user_id=f9d9a39675444fd9929465b27fb83cb3, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:28.570893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e659e7-1a9a-47d3-b942-4846bc07f4cb None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:28.780133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 640/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Sep 1 13:45:28 2022] PUT /identity/v3/projects/a745287849ba435e8bd1b543d58d9307/users/f9d9a39675444fd9929465b27fb83cb3/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:45:30.710735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd6c5d-66d9-4148-a6ce-e4325cad20b0 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:30.711311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd6c5d-66d9-4148-a6ce-e4325cad20b0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:30.711311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd6c5d-66d9-4148-a6ce-e4325cad20b0 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:30.713208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 634/1274] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:30.718484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488597-b25c-4c43-9369-3a1b02bc3e23 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:30.718898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488597-b25c-4c43-9369-3a1b02bc3e23 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:30.719010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488597-b25c-4c43-9369-3a1b02bc3e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:30.746093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e0488597-b25c-4c43-9369-3a1b02bc3e23 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:30.747351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e0488597-b25c-4c43-9369-3a1b02bc3e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:30.814913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 641/1275] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:31.086241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd6c5d-66d9-4148-a6ce-e4325cad20b0 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:31.189007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd6c5d-66d9-4148-a6ce-e4325cad20b0 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:31.206515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:31.224447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:31.225085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:31.225366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:31.225642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:31.236928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 13:45:31.242156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:31.243302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:31.244285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 635/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Sep 1 13:45:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:31.249460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488597-b25c-4c43-9369-3a1b02bc3e23 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:31.331177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:31.349453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488597-b25c-4c43-9369-3a1b02bc3e23 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:31.386056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:31.386662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:31.386937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:31.387205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:31.387867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:31.388170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:31.389513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:31.397490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 642/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Sep 1 13:45:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 13:45:31.403977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:31.517995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f740fb-e468-4cce-9ab9-bf49c6fa22e7 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:31.551949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:31.552576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:31.552852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:31.553122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:31.562834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:31.564282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:31.578925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:31.580181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:31.581123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:31.581123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:31.581123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:31.581123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 636/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:31.586303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:31.688504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daa476-ed6f-424f-9473-299a5f38f966 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:31.723351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:31.724043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:31.724236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:31.724514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:31.725167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:31.725474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:31.726912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:31.739085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 643/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:31.745165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:31.778085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:31.848113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb2dbc-abe3-4def-b87c-d57763f221b9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:31.884099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029507-adbf-4a04-afed-053709bb171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:31.884717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029507-adbf-4a04-afed-053709bb171b None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:31.884989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029507-adbf-4a04-afed-053709bb171b None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:31.885254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029507-adbf-4a04-afed-053709bb171b None admin] PATH_INFO: `/v3/users/placement` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:31.892237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46029507-adbf-4a04-afed-053709bb171b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:31.898195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46029507-adbf-4a04-afed-053709bb171b None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-46029507-adbf-4a04-afed-053709bb171b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 13:45:31.904862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:31.905808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:31.906829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 637/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:31.911782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:31.915600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:32.010174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9695e0-f795-4fd6-b6fd-58b94fb37475 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:32.043524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:32.044131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:32.044412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:32.044686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:32.045415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:32.045722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:32.047273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:32.066248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 644/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Sep 1 13:45:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:32.072675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029507-adbf-4a04-afed-053709bb171b None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:32.161980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:32.171993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029507-adbf-4a04-afed-053709bb171b None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:32.206143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:32.206755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:32.207037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:32.207313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:32.280877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:32.282340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:32.376366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:32.386736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 638/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Sep 1 13:45:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 13:45:32.392810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:32.456243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:32.496265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d715bc-47e3-464a-a6b8-1d6fc7b84ed2 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:32.537718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:32.537718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:32.537718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:32.537718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:32.544744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:32.546229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 13:45:32.561378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:32.572646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 13:45:32.573649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 13:45:32.573649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 13:45:32.573649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:32.573649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 645/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:32.573918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:32.669431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bee7ff-211e-4743-a06e-83d3bf985512 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:32.704335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:32.704939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:32.705210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:32.705477 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:32.706136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:32.706430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:32.707791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:32.719283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 639/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Sep 1 13:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:32.725270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:32.826808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a5165-6165-4159-a9b8-5f0dac9d7d92 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:32.841040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:32.862949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:32.863578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:32.863852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:32.864117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:32.870967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:32.872389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 13:45:32.884108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:32.891515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: ERROR keystone.server.flask.application Sep 01 13:45:32.892693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 646/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Sep 1 13:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:45:32.893294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:32.943497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:32.992178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873c9c-de75-4ce3-bdbd-c869683ab763 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:33.025569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31cd34-895f-4d1e-a826-69b75e58a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:33.026262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31cd34-895f-4d1e-a826-69b75e58a00f None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:33.027036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31cd34-895f-4d1e-a826-69b75e58a00f None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:33.027306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31cd34-895f-4d1e-a826-69b75e58a00f None admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:33.028038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31cd34-895f-4d1e-a826-69b75e58a00f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:33.028302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31cd34-895f-4d1e-a826-69b75e58a00f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:33.029731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31cd34-895f-4d1e-a826-69b75e58a00f None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:33.041415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 640/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Sep 1 13:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:33.047851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:33.147289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92a6e6-abf9-489a-833d-c9d88de9149d None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:33.180901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e805e7e-9f43-445c-bef7-7d1de1ff948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:33.181510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e805e7e-9f43-445c-bef7-7d1de1ff948f None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:33.181809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e805e7e-9f43-445c-bef7-7d1de1ff948f None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:33.182086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e805e7e-9f43-445c-bef7-7d1de1ff948f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:33.182868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e805e7e-9f43-445c-bef7-7d1de1ff948f None admin] RBAC: Adding query filter params (role.id=d1f9f95d9d0841e59b294cb7901942a1, scope.project.id=a745287849ba435e8bd1b543d58d9307, user.id=f9d9a39675444fd9929465b27fb83cb3) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:33.183174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e805e7e-9f43-445c-bef7-7d1de1ff948f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:33.184631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e805e7e-9f43-445c-bef7-7d1de1ff948f None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:33.190585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 647/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Sep 1 13:45:33 2022] GET /identity/v3/role_assignments?user.id=f9d9a39675444fd9929465b27fb83cb3&scope.project.id=a745287849ba435e8bd1b543d58d9307&role.id=d1f9f95d9d0841e59b294cb7901942a1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:35.152888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fe237-67e4-4bf8-8b38-443278eb06ad None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.153435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fe237-67e4-4bf8-8b38-443278eb06ad None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.153435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fe237-67e4-4bf8-8b38-443278eb06ad None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.155068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 641/1288] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:35.160637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321f171-8e3a-4b93-80f1-cc5ffe9c701d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.161165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321f171-8e3a-4b93-80f1-cc5ffe9c701d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.161297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321f171-8e3a-4b93-80f1-cc5ffe9c701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.181994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4321f171-8e3a-4b93-80f1-cc5ffe9c701d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:35.183213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4321f171-8e3a-4b93-80f1-cc5ffe9c701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:35.213235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:35.250459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 648/1289] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:35.520810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fe237-67e4-4bf8-8b38-443278eb06ad None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:35.632755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fe237-67e4-4bf8-8b38-443278eb06ad None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:35.643449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:35.667260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:35.667886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.668167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.668438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] PATH_INFO: `/v3/services/placement` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.669065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:35.670461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:35.679155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:35.682265 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:35.683316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 642/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:45:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:35.686525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321f171-8e3a-4b93-80f1-cc5ffe9c701d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:35.784164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321f171-8e3a-4b93-80f1-cc5ffe9c701d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:35.818716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b063b0-f59e-4438-9af0-1145dda78094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:35.819316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b063b0-f59e-4438-9af0-1145dda78094 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.819594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b063b0-f59e-4438-9af0-1145dda78094 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.819863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b063b0-f59e-4438-9af0-1145dda78094 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.820492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b063b0-f59e-4438-9af0-1145dda78094 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:35.820819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b063b0-f59e-4438-9af0-1145dda78094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:35.822219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b063b0-f59e-4438-9af0-1145dda78094 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:35.829310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 649/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Sep 1 13:45:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:35.835027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:35.942236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169e3d-fff1-4dd0-acbd-5772f3469dfe None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:35.977689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274e647-7c74-4a95-bd95-ca5831db3386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:35.978396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274e647-7c74-4a95-bd95-ca5831db3386 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:35.978678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274e647-7c74-4a95-bd95-ca5831db3386 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:35.978946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274e647-7c74-4a95-bd95-ca5831db3386 None admin] PATH_INFO: `/v3/services` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:35.979576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274e647-7c74-4a95-bd95-ca5831db3386 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:35.979875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274e647-7c74-4a95-bd95-ca5831db3386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:35.981252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274e647-7c74-4a95-bd95-ca5831db3386 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:35.986111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 643/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Sep 1 13:45:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:37.970145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3e93-6499-49c1-ba35-a06244915669 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:37.970704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3e93-6499-49c1-ba35-a06244915669 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:37.970704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d3e93-6499-49c1-ba35-a06244915669 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:37.972029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 650/1293] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:45:37.977798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a86ba-ab32-4da2-8250-ff8011e52c39 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:37.982041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a86ba-ab32-4da2-8250-ff8011e52c39 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:37.982041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a86ba-ab32-4da2-8250-ff8011e52c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:37.998601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-728a86ba-ab32-4da2-8250-ff8011e52c39 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:37.999881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-728a86ba-ab32-4da2-8250-ff8011e52c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:38.032060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:38.072872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 644/1294] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:38.343689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3e93-6499-49c1-ba35-a06244915669 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:38.376527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:38.451216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d3e93-6499-49c1-ba35-a06244915669 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:38.487238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08422ad2-a940-4c99-8181-44dbae928276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:38.487843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08422ad2-a940-4c99-8181-44dbae928276 None admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:38.488119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08422ad2-a940-4c99-8181-44dbae928276 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:38.488388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08422ad2-a940-4c99-8181-44dbae928276 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:38.489049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08422ad2-a940-4c99-8181-44dbae928276 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:38.490429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08422ad2-a940-4c99-8181-44dbae928276 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:38.645415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 651/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Sep 1 13:45:38 2022] POST /identity/v3/services => generated 246 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:45:40.618812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef644cc8-0529-4b0d-a9c7-2b6cbbd6af26 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:40.619637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef644cc8-0529-4b0d-a9c7-2b6cbbd6af26 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:40.619637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef644cc8-0529-4b0d-a9c7-2b6cbbd6af26 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:40.620686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 645/1296] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:45:40.626517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2a70f-90ff-4361-82c6-abb266d69138 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:40.627046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2a70f-90ff-4361-82c6-abb266d69138 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:40.627046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2a70f-90ff-4361-82c6-abb266d69138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:40.651244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-77b2a70f-90ff-4361-82c6-abb266d69138 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:40.654978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-77b2a70f-90ff-4361-82c6-abb266d69138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:40.744206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 652/1297] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 13:45:41.016835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef644cc8-0529-4b0d-a9c7-2b6cbbd6af26 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.129877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef644cc8-0529-4b0d-a9c7-2b6cbbd6af26 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.166174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.166811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.167081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.167344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.167930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.169299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:41.178927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:41.181062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:41.183166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 646/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 13:45:41.188548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2a70f-90ff-4361-82c6-abb266d69138 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.209682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:41.299707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2a70f-90ff-4361-82c6-abb266d69138 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.342221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.342850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.343133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.343408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.344062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:41.344380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.345797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.351895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 653/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:41.358399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.396668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:41.477581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29cb09-1b54-4499-baa7-d54cbff94ac9 None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.515145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb02793-4ec7-4cab-b62f-a8ed78e46203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.515926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb02793-4ec7-4cab-b62f-a8ed78e46203 None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.516313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb02793-4ec7-4cab-b62f-a8ed78e46203 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.516678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb02793-4ec7-4cab-b62f-a8ed78e46203 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.517484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb02793-4ec7-4cab-b62f-a8ed78e46203 None admin] RBAC: Adding query filter params (interface=public, service_id=bbfd10bad6ca4a6f82064848950b90fa, region_id=RegionOne) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:41.517935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb02793-4ec7-4cab-b62f-a8ed78e46203 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.519410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb02793-4ec7-4cab-b62f-a8ed78e46203 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.526070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 647/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/endpoints?service_id=bbfd10bad6ca4a6f82064848950b90fa&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:41.535540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:41.630093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:41.650637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa1395-6ae7-49da-8361-37974c112e95 None admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:41.687433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30514b71-e154-44e9-84c8-f2730e6dbe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:41.688044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30514b71-e154-44e9-84c8-f2730e6dbe62 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:41.688319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30514b71-e154-44e9-84c8-f2730e6dbe62 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:41.688586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30514b71-e154-44e9-84c8-f2730e6dbe62 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:41.689196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30514b71-e154-44e9-84c8-f2730e6dbe62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:41.690586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30514b71-e154-44e9-84c8-f2730e6dbe62 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:41.698334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 654/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Sep 1 13:45:41 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:45:43.640852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571659f2-634a-4dab-8314-d323f051ffcc None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:43.641432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571659f2-634a-4dab-8314-d323f051ffcc None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:43.641432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571659f2-634a-4dab-8314-d323f051ffcc None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:43.642848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 648/1302] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:45:43.649479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5bd6e-c7e2-48cc-8736-03b40fb2113e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:43.650367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5bd6e-c7e2-48cc-8736-03b40fb2113e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:43.650367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5bd6e-c7e2-48cc-8736-03b40fb2113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:43.668427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-fbb5bd6e-c7e2-48cc-8736-03b40fb2113e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:45:43.674371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fbb5bd6e-c7e2-48cc-8736-03b40fb2113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:45:43.745523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 655/1303] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:45:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 13:45:44.032492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571659f2-634a-4dab-8314-d323f051ffcc None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.202980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:44.215518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571659f2-634a-4dab-8314-d323f051ffcc None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:44.281729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:44.282646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:44.283007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:44.283387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] PATH_INFO: `/v3/services/placement` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:44.284077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:44.285649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 13:45:44.296288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 13:45:44.297690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application with Lock( Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self._enter() Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application return self.creator() Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application created_value = creator( Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: ERROR keystone.server.flask.application Sep 01 13:45:44.303017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 649/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Sep 1 13:45:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 13:45:44.305863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5bd6e-c7e2-48cc-8736-03b40fb2113e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.392576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:44.417752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5bd6e-c7e2-48cc-8736-03b40fb2113e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:44.454196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebc7f1-653c-45c9-a555-5e774a150bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:44.454812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebc7f1-653c-45c9-a555-5e774a150bc6 None admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:44.455087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebc7f1-653c-45c9-a555-5e774a150bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:44.455353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebc7f1-653c-45c9-a555-5e774a150bc6 None admin] PATH_INFO: `/v3/services` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:44.455997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebc7f1-653c-45c9-a555-5e774a150bc6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:45:44.456292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebc7f1-653c-45c9-a555-5e774a150bc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:44.457649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebc7f1-653c-45c9-a555-5e774a150bc6 None admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:44.463277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 656/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Sep 1 13:45:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:45:44.469323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:45:44.580766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a5142-e613-4acf-95c3-7a7222432fbe None admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:45:44.619863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45015781-957b-49ae-afe2-291f716e20c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:45:44.620610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45015781-957b-49ae-afe2-291f716e20c3 None admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:45:44.620980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45015781-957b-49ae-afe2-291f716e20c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:45:44.621349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45015781-957b-49ae-afe2-291f716e20c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:45:44.622178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45015781-957b-49ae-afe2-291f716e20c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:45:44.623732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45015781-957b-49ae-afe2-291f716e20c3 None admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:45:44.729231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 650/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:45:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:45:54.510029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=30 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:45:54.723534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:08.158657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1294c84-1c4e-4743-abd3-2f1b0532126b None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:08.158657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1294c84-1c4e-4743-abd3-2f1b0532126b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:08.158657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1294c84-1c4e-4743-abd3-2f1b0532126b None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:08.164845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5163-fdf8-46fd-8425-6b96783924ea None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:08.165244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5163-fdf8-46fd-8425-6b96783924ea None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:08.165406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5163-fdf8-46fd-8425-6b96783924ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:08.168275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 657/1307] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:08.260490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3a0a5163-fdf8-46fd-8425-6b96783924ea None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:08.261858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3a0a5163-fdf8-46fd-8425-6b96783924ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:08.398448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 651/1308] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:52:08.690667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47167afe-84d9-4ea3-885e-9c770151070e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:08.691351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47167afe-84d9-4ea3-885e-9c770151070e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:08.691855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47167afe-84d9-4ea3-885e-9c770151070e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:08.693458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 658/1309] 10.222.0.61 () {54 vars in 965 bytes} [Thu Sep 1 13:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:08.701010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f2ba-363d-46ad-bad1-bfecc17de5a9 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:08.701730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f2ba-363d-46ad-bad1-bfecc17de5a9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:08.705331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f2ba-363d-46ad-bad1-bfecc17de5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:08.796568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-bcf7f2ba-363d-46ad-bad1-bfecc17de5a9 None None] MFA Rules not processed for user `b1c427a987ba48878873734ecf0f86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:08.798274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bcf7f2ba-363d-46ad-bad1-bfecc17de5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:08.821490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:08.901701 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 652/1310] 10.222.0.61 () {58 vars in 1062 bytes} [Thu Sep 1 13:52:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:08.912026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47167afe-84d9-4ea3-885e-9c770151070e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:08.913812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-47167afe-84d9-4ea3-885e-9c770151070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:09.064876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47167afe-84d9-4ea3-885e-9c770151070e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:09.163098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed836f7-0b9a-4720-98a9-b282e086855b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:09.164027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed836f7-0b9a-4720-98a9-b282e086855b service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:09.164314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed836f7-0b9a-4720-98a9-b282e086855b service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:09.164582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed836f7-0b9a-4720-98a9-b282e086855b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:09.166391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fed836f7-0b9a-4720-98a9-b282e086855b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:09.232554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:09.304614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed836f7-0b9a-4720-98a9-b282e086855b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:09.306091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed836f7-0b9a-4720-98a9-b282e086855b service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:09.330420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 659/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:11.615167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc04b9-373e-441d-98b0-425f903dcabb None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:11.615604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc04b9-373e-441d-98b0-425f903dcabb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:11.615698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc04b9-373e-441d-98b0-425f903dcabb None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:11.617062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 653/1312] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:11.623279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f984ef-e775-45c8-9c8a-a9e3713b25cb None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:11.623731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f984ef-e775-45c8-9c8a-a9e3713b25cb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:11.623840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f984ef-e775-45c8-9c8a-a9e3713b25cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:11.713781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-02f984ef-e775-45c8-9c8a-a9e3713b25cb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:11.715027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-02f984ef-e775-45c8-9c8a-a9e3713b25cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:11.817797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 660/1313] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:12.102858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc04b9-373e-441d-98b0-425f903dcabb None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:12.104447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-23bc04b9-373e-441d-98b0-425f903dcabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:12.208283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc04b9-373e-441d-98b0-425f903dcabb None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:12.280022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:12.282601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aef97b-6920-4080-a216-1ee2951d9a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:12.283213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef97b-6920-4080-a216-1ee2951d9a4a service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:12.283491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef97b-6920-4080-a216-1ee2951d9a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:12.283761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef97b-6920-4080-a216-1ee2951d9a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:12.285395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-65aef97b-6920-4080-a216-1ee2951d9a4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:12.398085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aef97b-6920-4080-a216-1ee2951d9a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:12.399568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aef97b-6920-4080-a216-1ee2951d9a4a service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:12.419197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 654/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:14.895843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6120d-efbf-4085-97fe-d716fc0e1834 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:14.896427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6120d-efbf-4085-97fe-d716fc0e1834 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:14.896427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6120d-efbf-4085-97fe-d716fc0e1834 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:14.897756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 661/1315] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:14.903591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f002b6-3976-4336-8fb7-d2bc614996b6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:14.904009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f002b6-3976-4336-8fb7-d2bc614996b6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:14.904139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f002b6-3976-4336-8fb7-d2bc614996b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:14.971017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-47f002b6-3976-4336-8fb7-d2bc614996b6 None None] MFA Rules not processed for user `b4e7b7be35104fb680b3ffa073772913`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:14.972343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-47f002b6-3976-4336-8fb7-d2bc614996b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:15.029256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:15.075311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 655/1316] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:15.360314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6120d-efbf-4085-97fe-d716fc0e1834 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:15.361886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-29e6120d-efbf-4085-97fe-d716fc0e1834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:15.372131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:15.470887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6120d-efbf-4085-97fe-d716fc0e1834 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:15.545268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af97fe8-06b0-441f-9430-c3445d4f4412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:15.545905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af97fe8-06b0-441f-9430-c3445d4f4412 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:15.546188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af97fe8-06b0-441f-9430-c3445d4f4412 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:15.546458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af97fe8-06b0-441f-9430-c3445d4f4412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:15.548091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1af97fe8-06b0-441f-9430-c3445d4f4412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:15.612879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:15.662739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af97fe8-06b0-441f-9430-c3445d4f4412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:15.664207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af97fe8-06b0-441f-9430-c3445d4f4412 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:15.683254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 662/1317] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:18.373918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2db6c-de0c-42b0-ba8a-ee7f243806dd None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:18.374433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2db6c-de0c-42b0-ba8a-ee7f243806dd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:18.374527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2db6c-de0c-42b0-ba8a-ee7f243806dd None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:18.375807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 656/1318] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:18.381030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85129d-df47-4521-a102-a7fb0aa1f823 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:18.381442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85129d-df47-4521-a102-a7fb0aa1f823 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:18.381571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85129d-df47-4521-a102-a7fb0aa1f823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:18.446604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-da85129d-df47-4521-a102-a7fb0aa1f823 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:18.447809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-da85129d-df47-4521-a102-a7fb0aa1f823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:18.544202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 663/1319] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:18.825714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2db6c-de0c-42b0-ba8a-ee7f243806dd None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:18.827289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-10b2db6c-de0c-42b0-ba8a-ee7f243806dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:18.930438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2db6c-de0c-42b0-ba8a-ee7f243806dd None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:19.005167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef7a61-da38-444d-888a-48f162b9b290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:19.006023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef7a61-da38-444d-888a-48f162b9b290 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:19.006159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef7a61-da38-444d-888a-48f162b9b290 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:19.006476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef7a61-da38-444d-888a-48f162b9b290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:19.008011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-32ef7a61-da38-444d-888a-48f162b9b290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:19.016486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:19.117869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef7a61-da38-444d-888a-48f162b9b290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:19.119312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef7a61-da38-444d-888a-48f162b9b290 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:19.138055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 657/1320] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:22.148493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1b0b5-4a62-44ea-b0dc-48fbac1e8047 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:22.148990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1b0b5-4a62-44ea-b0dc-48fbac1e8047 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:22.149103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1b0b5-4a62-44ea-b0dc-48fbac1e8047 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:22.150542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 664/1321] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:22.155742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf549a-5f4d-4410-8fea-9adc7b3ec1be None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:22.156194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf549a-5f4d-4410-8fea-9adc7b3ec1be None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:22.156291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf549a-5f4d-4410-8fea-9adc7b3ec1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:22.224690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-9aaf549a-5f4d-4410-8fea-9adc7b3ec1be None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:22.225967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9aaf549a-5f4d-4410-8fea-9adc7b3ec1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:22.284587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:22.324774 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 658/1322] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:22.607761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1b0b5-4a62-44ea-b0dc-48fbac1e8047 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:22.609414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-00b1b0b5-4a62-44ea-b0dc-48fbac1e8047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:22.632683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:22.723543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1b0b5-4a62-44ea-b0dc-48fbac1e8047 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:22.802416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902e54b-96a2-472a-bd67-ee848aa0f5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:22.803044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902e54b-96a2-472a-bd67-ee848aa0f5ca service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:22.803326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902e54b-96a2-472a-bd67-ee848aa0f5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:22.803601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902e54b-96a2-472a-bd67-ee848aa0f5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:22.805312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1902e54b-96a2-472a-bd67-ee848aa0f5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:22.878092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:22.916304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902e54b-96a2-472a-bd67-ee848aa0f5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:22.917730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902e54b-96a2-472a-bd67-ee848aa0f5ca service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:22.936010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 665/1323] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:24.940678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3fd37-c32b-4881-a2e7-b4aecba42018 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:24.941211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3fd37-c32b-4881-a2e7-b4aecba42018 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:24.941211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3fd37-c32b-4881-a2e7-b4aecba42018 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:24.942636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 659/1324] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:24.948269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012f5c3-02c0-455a-b60f-249a6ee98152 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:24.948749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012f5c3-02c0-455a-b60f-249a6ee98152 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:24.948953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012f5c3-02c0-455a-b60f-249a6ee98152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:25.019721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a012f5c3-02c0-455a-b60f-249a6ee98152 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:25.020950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a012f5c3-02c0-455a-b60f-249a6ee98152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:25.120350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 666/1325] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:25.410691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3fd37-c32b-4881-a2e7-b4aecba42018 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:25.412397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-47c3fd37-c32b-4881-a2e7-b4aecba42018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:25.531222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3fd37-c32b-4881-a2e7-b4aecba42018 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:25.613687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216bb7a-15d9-4bb0-9402-4b676dfef7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:25.614333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216bb7a-15d9-4bb0-9402-4b676dfef7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:25.614624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216bb7a-15d9-4bb0-9402-4b676dfef7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:25.614887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216bb7a-15d9-4bb0-9402-4b676dfef7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:25.616485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6216bb7a-15d9-4bb0-9402-4b676dfef7d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:25.634607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:25.735469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216bb7a-15d9-4bb0-9402-4b676dfef7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:25.736980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216bb7a-15d9-4bb0-9402-4b676dfef7d4 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:25.756694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 660/1326] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:27.738692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c1165-7061-4bbf-8c18-1617ff21eff2 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:27.739218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c1165-7061-4bbf-8c18-1617ff21eff2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:27.739218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c1165-7061-4bbf-8c18-1617ff21eff2 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:27.740633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 667/1327] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:27.745921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944469e-6340-40bc-afd8-03a0beec65a4 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:27.746855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944469e-6340-40bc-afd8-03a0beec65a4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:27.747134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944469e-6340-40bc-afd8-03a0beec65a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:27.813732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2944469e-6340-40bc-afd8-03a0beec65a4 None None] MFA Rules not processed for user `b4e7b7be35104fb680b3ffa073772913`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:27.814970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2944469e-6340-40bc-afd8-03a0beec65a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:27.888281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:27.920944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 661/1328] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:52:28.209575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c1165-7061-4bbf-8c18-1617ff21eff2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:28.211163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fa4c1165-7061-4bbf-8c18-1617ff21eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:28.243007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:28.318799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c1165-7061-4bbf-8c18-1617ff21eff2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:28.416973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58c8a2-d297-4b13-904f-673a6dcd33af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:28.417587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c8a2-d297-4b13-904f-673a6dcd33af service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:28.417896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c8a2-d297-4b13-904f-673a6dcd33af service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:28.418163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c8a2-d297-4b13-904f-673a6dcd33af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:28.420324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8b58c8a2-d297-4b13-904f-673a6dcd33af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:28.503527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:28.536300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58c8a2-d297-4b13-904f-673a6dcd33af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:28.537738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58c8a2-d297-4b13-904f-673a6dcd33af service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:28.557037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 668/1329] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:30.944576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5510-6f23-4d0c-93fa-d3b8b601be2a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:30.945111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5510-6f23-4d0c-93fa-d3b8b601be2a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:30.945111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5510-6f23-4d0c-93fa-d3b8b601be2a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:30.946552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 662/1330] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:30.952737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760b04f-71b9-45d7-a99a-2ed727dc21a2 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:30.953219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760b04f-71b9-45d7-a99a-2ed727dc21a2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:30.953326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760b04f-71b9-45d7-a99a-2ed727dc21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:31.026625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a760b04f-71b9-45d7-a99a-2ed727dc21a2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:31.027827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a760b04f-71b9-45d7-a99a-2ed727dc21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:31.135758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 669/1331] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:31.423096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5510-6f23-4d0c-93fa-d3b8b601be2a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:31.424702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1b8e5510-6f23-4d0c-93fa-d3b8b601be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:31.528471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5510-6f23-4d0c-93fa-d3b8b601be2a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:31.603108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38576e-955c-4588-9add-9b21e9533ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:31.603799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38576e-955c-4588-9add-9b21e9533ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:31.604079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38576e-955c-4588-9add-9b21e9533ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:31.604353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38576e-955c-4588-9add-9b21e9533ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:31.606581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1a38576e-955c-4588-9add-9b21e9533ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:31.634078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:31.714060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38576e-955c-4588-9add-9b21e9533ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:31.716000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38576e-955c-4588-9add-9b21e9533ab8 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:31.734256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 663/1332] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:33.724128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041cf76-600c-4fd5-b1df-85e1f877c864 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:33.724695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041cf76-600c-4fd5-b1df-85e1f877c864 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:33.724695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041cf76-600c-4fd5-b1df-85e1f877c864 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:33.726077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 670/1333] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:33.731159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1be50-7ce9-4f0d-b938-dd655e6cf66d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:33.731545 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1be50-7ce9-4f0d-b938-dd655e6cf66d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:33.731706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1be50-7ce9-4f0d-b938-dd655e6cf66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:33.797456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-0ef1be50-7ce9-4f0d-b938-dd655e6cf66d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:33.798690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0ef1be50-7ce9-4f0d-b938-dd655e6cf66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:33.875395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:33.897204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 664/1334] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:34.180713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041cf76-600c-4fd5-b1df-85e1f877c864 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:34.182296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7041cf76-600c-4fd5-b1df-85e1f877c864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:34.223274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:34.288373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041cf76-600c-4fd5-b1df-85e1f877c864 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:34.362295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a2641-d0a1-4f5a-b66d-36b0a17fc8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:34.362902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a2641-d0a1-4f5a-b66d-36b0a17fc8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:34.363176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a2641-d0a1-4f5a-b66d-36b0a17fc8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:34.363448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a2641-d0a1-4f5a-b66d-36b0a17fc8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:34.365045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d39a2641-d0a1-4f5a-b66d-36b0a17fc8d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:34.448382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:34.474983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a2641-d0a1-4f5a-b66d-36b0a17fc8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:34.476420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a2641-d0a1-4f5a-b66d-36b0a17fc8d2 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:34.495342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 671/1335] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:37.077977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9a75-daac-4ef4-91f0-c47ecdd35ed0 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:37.078552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9a75-daac-4ef4-91f0-c47ecdd35ed0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:37.078552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9a75-daac-4ef4-91f0-c47ecdd35ed0 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:37.079892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 665/1336] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:37.085416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52a236-6288-4aa3-822a-aee438094586 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:37.085958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52a236-6288-4aa3-822a-aee438094586 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:37.085958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52a236-6288-4aa3-822a-aee438094586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:37.151381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-dd52a236-6288-4aa3-822a-aee438094586 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:37.153119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dd52a236-6288-4aa3-822a-aee438094586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:37.254886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 672/1337] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:52:37.556262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c9a75-daac-4ef4-91f0-c47ecdd35ed0 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:37.562454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7d5c9a75-daac-4ef4-91f0-c47ecdd35ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:37.663591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c9a75-daac-4ef4-91f0-c47ecdd35ed0 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:37.738203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef978f-3615-46e0-8202-a09e265dd06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:37.738848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef978f-3615-46e0-8202-a09e265dd06a service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:37.739125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef978f-3615-46e0-8202-a09e265dd06a service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:37.739389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef978f-3615-46e0-8202-a09e265dd06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:37.741007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-27ef978f-3615-46e0-8202-a09e265dd06a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:37.777659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:37.857435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef978f-3615-46e0-8202-a09e265dd06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:37.859035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef978f-3615-46e0-8202-a09e265dd06a service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:37.880222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 666/1338] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:52:41.993500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df2b39-4854-427c-a8ee-988860c44817 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:41.994132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df2b39-4854-427c-a8ee-988860c44817 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:41.994132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df2b39-4854-427c-a8ee-988860c44817 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:41.995542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 673/1339] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:42.001238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c831-b240-434c-b0bf-0ba5f314ef13 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:42.001948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c831-b240-434c-b0bf-0ba5f314ef13 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:42.001948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c831-b240-434c-b0bf-0ba5f314ef13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:42.067856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6f95c831-b240-434c-b0bf-0ba5f314ef13 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:42.069074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6f95c831-b240-434c-b0bf-0ba5f314ef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:42.157519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:42.169663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 667/1340] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:42.456607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df2b39-4854-427c-a8ee-988860c44817 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:42.458270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d9df2b39-4854-427c-a8ee-988860c44817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:42.508252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:42.567076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df2b39-4854-427c-a8ee-988860c44817 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:42.647775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834caf-d624-4dda-9081-6041cfb80bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:42.648387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834caf-d624-4dda-9081-6041cfb80bed service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:42.648676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834caf-d624-4dda-9081-6041cfb80bed service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:42.648956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834caf-d624-4dda-9081-6041cfb80bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:42.650606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-be834caf-d624-4dda-9081-6041cfb80bed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:42.748963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:42.765211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834caf-d624-4dda-9081-6041cfb80bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:42.766687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834caf-d624-4dda-9081-6041cfb80bed service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:42.785530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 674/1341] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:45.814384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e7810-3056-43b7-be2f-34a38ae4717e None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:45.814929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e7810-3056-43b7-be2f-34a38ae4717e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:45.814929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e7810-3056-43b7-be2f-34a38ae4717e None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:45.816272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 668/1342] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:45.821683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011aef1a-85fc-480b-bd11-65f89de5bb91 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:45.822187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011aef1a-85fc-480b-bd11-65f89de5bb91 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:45.822532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011aef1a-85fc-480b-bd11-65f89de5bb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:45.891462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-011aef1a-85fc-480b-bd11-65f89de5bb91 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:45.892668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-011aef1a-85fc-480b-bd11-65f89de5bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:45.990813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 675/1343] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:46.289049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e171e6-1df7-49fa-a80a-423d654396ec None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:46.289338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e171e6-1df7-49fa-a80a-423d654396ec None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:46.289612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e171e6-1df7-49fa-a80a-423d654396ec None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:46.291009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 669/1344] 10.222.0.61 () {54 vars in 965 bytes} [Thu Sep 1 13:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:46.297655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af13c-a1f4-418f-8602-3b661c477bfc None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:46.297977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af13c-a1f4-418f-8602-3b661c477bfc None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:46.298256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af13c-a1f4-418f-8602-3b661c477bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:46.370852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:46.380795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-934af13c-a1f4-418f-8602-3b661c477bfc None None] MFA Rules not processed for user `b1c427a987ba48878873734ecf0f86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:46.382071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-934af13c-a1f4-418f-8602-3b661c477bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:46.463580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 676/1345] 10.222.0.61 () {58 vars in 1062 bytes} [Thu Sep 1 13:52:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:46.473060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e171e6-1df7-49fa-a80a-423d654396ec None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:46.474655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-55e171e6-1df7-49fa-a80a-423d654396ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:46.588457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e171e6-1df7-49fa-a80a-423d654396ec None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:46.664377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94422615-3e87-490f-bd33-4190f362b235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:46.665077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94422615-3e87-490f-bd33-4190f362b235 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:46.665427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94422615-3e87-490f-bd33-4190f362b235 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:46.665619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94422615-3e87-490f-bd33-4190f362b235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:46.667309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-94422615-3e87-490f-bd33-4190f362b235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:46.710422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:46.778034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94422615-3e87-490f-bd33-4190f362b235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:46.779489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94422615-3e87-490f-bd33-4190f362b235 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:46.798633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 670/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:51.701418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37786832-37b0-4ca5-a0b9-4e8836ef677e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:51.701936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37786832-37b0-4ca5-a0b9-4e8836ef677e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:51.702124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37786832-37b0-4ca5-a0b9-4e8836ef677e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:51.703425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 677/1347] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:51.708988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1f070-dcbf-4f2d-818b-cee981b6350c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:51.709526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1f070-dcbf-4f2d-818b-cee981b6350c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:51.709791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1f070-dcbf-4f2d-818b-cee981b6350c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:51.782136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-ebd1f070-dcbf-4f2d-818b-cee981b6350c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:51.783272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ebd1f070-dcbf-4f2d-818b-cee981b6350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:51.887824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:51.890714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 671/1348] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:52:52.175884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37786832-37b0-4ca5-a0b9-4e8836ef677e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:52:52.177447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-37786832-37b0-4ca5-a0b9-4e8836ef677e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:52.297628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37786832-37b0-4ca5-a0b9-4e8836ef677e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:52:52.354311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:52.381490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bcc6c-0b28-44c8-a5e9-2ae15799230d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:52:52.382128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bcc6c-0b28-44c8-a5e9-2ae15799230d service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:52.382419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bcc6c-0b28-44c8-a5e9-2ae15799230d service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:52.382692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bcc6c-0b28-44c8-a5e9-2ae15799230d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:52.384361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2e4bcc6c-0b28-44c8-a5e9-2ae15799230d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:52.497443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bcc6c-0b28-44c8-a5e9-2ae15799230d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:52:52.498958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bcc6c-0b28-44c8-a5e9-2ae15799230d service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:52:52.517365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 678/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:52:57.446572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b01ae1-8c5c-418f-92ee-ff9f969691d7 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:57.446572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b01ae1-8c5c-418f-92ee-ff9f969691d7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:57.446572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b01ae1-8c5c-418f-92ee-ff9f969691d7 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:57.447757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 672/1350] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 13:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:52:57.457370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0ffce-7096-425c-850b-b0af3298d6c3 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:52:57.457370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0ffce-7096-425c-850b-b0af3298d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:52:57.457370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0ffce-7096-425c-850b-b0af3298d6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:52:57.588564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-cca0ffce-7096-425c-850b-b0af3298d6c3 None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:52:57.590614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cca0ffce-7096-425c-850b-b0af3298d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:52:57.618448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:52:57.721591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 679/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 13:52:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:01.983663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93006a-f511-4a0d-b840-424bc6686174 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:01.984272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93006a-f511-4a0d-b840-424bc6686174 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:01.984272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93006a-f511-4a0d-b840-424bc6686174 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:01.985606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 673/1352] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 13:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:01.990741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad2c82-6a4c-4167-b179-ded77dd6bf07 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:01.991153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad2c82-6a4c-4167-b179-ded77dd6bf07 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:01.991292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad2c82-6a4c-4167-b179-ded77dd6bf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:02.069760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-38ad2c82-6a4c-4167-b179-ded77dd6bf07 None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:02.071016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-38ad2c82-6a4c-4167-b179-ded77dd6bf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:02.149932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:02.188280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 680/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 13:53:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:03.646394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5315a6-1e10-44b5-b161-66f72d73e461 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:03.646936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5315a6-1e10-44b5-b161-66f72d73e461 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:03.648411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5315a6-1e10-44b5-b161-66f72d73e461 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:03.649899 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 674/1354] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 13:53:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:03.655867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cdf09-6307-4bb9-873e-7eb801aacf4e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:03.656412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cdf09-6307-4bb9-873e-7eb801aacf4e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:03.656526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cdf09-6307-4bb9-873e-7eb801aacf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:03.737130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-556cdf09-6307-4bb9-873e-7eb801aacf4e None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:03.738417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-556cdf09-6307-4bb9-873e-7eb801aacf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:03.850722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 681/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 13:53:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:53:06.993339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073fdb7-274b-4c5c-a250-a3173e1c3c3f None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:06.993866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073fdb7-274b-4c5c-a250-a3173e1c3c3f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:06.994051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073fdb7-274b-4c5c-a250-a3173e1c3c3f None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:06.995884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 675/1356] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:07.001554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40e9e-b925-491c-8434-145eeed769a3 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:07.001967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40e9e-b925-491c-8434-145eeed769a3 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:07.002135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40e9e-b925-491c-8434-145eeed769a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:07.045815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:07.082073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d6d40e9e-b925-491c-8434-145eeed769a3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:07.083275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d6d40e9e-b925-491c-8434-145eeed769a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:07.218366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 682/1357] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:07.529951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00140ed4-69b8-4054-9e08-e537eb65e0d4 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:07.530566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00140ed4-69b8-4054-9e08-e537eb65e0d4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:07.531065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00140ed4-69b8-4054-9e08-e537eb65e0d4 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:07.533105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 676/1358] 10.222.0.61 () {54 vars in 962 bytes} [Thu Sep 1 13:53:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:07.545683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893beb0e-fc2d-4622-b79c-e6b938a9afaa None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:07.545683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893beb0e-fc2d-4622-b79c-e6b938a9afaa None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:07.545683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893beb0e-fc2d-4622-b79c-e6b938a9afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:07.633203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-893beb0e-fc2d-4622-b79c-e6b938a9afaa None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:07.634599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-893beb0e-fc2d-4622-b79c-e6b938a9afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:07.692833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:07.771345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 683/1359] 10.222.0.61 () {58 vars in 1059 bytes} [Thu Sep 1 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:07.779735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00140ed4-69b8-4054-9e08-e537eb65e0d4 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:07.781315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-00140ed4-69b8-4054-9e08-e537eb65e0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:08.081015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00140ed4-69b8-4054-9e08-e537eb65e0d4 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:08.163947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:08.218192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ca8b2-d2ff-426c-960a-458d97929a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:08.219442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca8b2-d2ff-426c-960a-458d97929a90 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:08.219986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca8b2-d2ff-426c-960a-458d97929a90 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:08.220629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca8b2-d2ff-426c-960a-458d97929a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:08.223133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6d9ca8b2-d2ff-426c-960a-458d97929a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:08.345233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ca8b2-d2ff-426c-960a-458d97929a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:08.347212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ca8b2-d2ff-426c-960a-458d97929a90 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:08.372973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 677/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:53:10.526940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73307cbd-4169-4dd2-a732-208c96ab5117 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:10.527460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73307cbd-4169-4dd2-a732-208c96ab5117 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:10.527460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73307cbd-4169-4dd2-a732-208c96ab5117 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:10.528932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 684/1361] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:10.534201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61088648-fa93-4899-8271-718f0947dc54 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:10.534688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61088648-fa93-4899-8271-718f0947dc54 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:10.534793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61088648-fa93-4899-8271-718f0947dc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:10.609558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-61088648-fa93-4899-8271-718f0947dc54 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:10.622052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-61088648-fa93-4899-8271-718f0947dc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:10.653225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:10.733941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 678/1362] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:11.017809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73307cbd-4169-4dd2-a732-208c96ab5117 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:11.019331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-73307cbd-4169-4dd2-a732-208c96ab5117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:11.175783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73307cbd-4169-4dd2-a732-208c96ab5117 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:11.188564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:11.338337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441cd3e-06b4-41b2-8f26-baa61a03efa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:11.339210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cd3e-06b4-41b2-8f26-baa61a03efa7 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:11.339488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cd3e-06b4-41b2-8f26-baa61a03efa7 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:11.339797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cd3e-06b4-41b2-8f26-baa61a03efa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:11.341369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a441cd3e-06b4-41b2-8f26-baa61a03efa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:11.443449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441cd3e-06b4-41b2-8f26-baa61a03efa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:11.445157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441cd3e-06b4-41b2-8f26-baa61a03efa7 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:11.468234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 685/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:13.832938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8e12-6c4b-4809-8eb1-9f198dab2edb None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:13.833485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8e12-6c4b-4809-8eb1-9f198dab2edb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:13.833485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8e12-6c4b-4809-8eb1-9f198dab2edb None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:13.834854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 679/1364] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:13.840732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624433cc-179a-4036-979b-4d49ca269fbe None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:13.841174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624433cc-179a-4036-979b-4d49ca269fbe None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:13.841275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624433cc-179a-4036-979b-4d49ca269fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:13.858029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:13.919511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-624433cc-179a-4036-979b-4d49ca269fbe None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:13.920720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-624433cc-179a-4036-979b-4d49ca269fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:14.022580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 686/1365] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:14.323952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65ffc0-7dd1-4834-8c15-e23bccb0029e None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:14.324273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65ffc0-7dd1-4834-8c15-e23bccb0029e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:14.324545 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65ffc0-7dd1-4834-8c15-e23bccb0029e None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:14.326688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 680/1366] 10.222.0.61 () {54 vars in 962 bytes} [Thu Sep 1 13:53:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:14.331853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3908e7-f902-46d2-86dd-6c35dee9ce41 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:14.332379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3908e7-f902-46d2-86dd-6c35dee9ce41 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:14.332718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3908e7-f902-46d2-86dd-6c35dee9ce41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:14.416391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-cd3908e7-f902-46d2-86dd-6c35dee9ce41 None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:14.417715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cd3908e7-f902-46d2-86dd-6c35dee9ce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:14.456736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:14.546911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 687/1367] 10.222.0.61 () {58 vars in 1059 bytes} [Thu Sep 1 13:53:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:14.554747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65ffc0-7dd1-4834-8c15-e23bccb0029e None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:14.556673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3b65ffc0-7dd1-4834-8c15-e23bccb0029e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:14.726460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65ffc0-7dd1-4834-8c15-e23bccb0029e None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:14.754127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:14.858332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9358564-ba46-4dda-8011-4a8d5db0a9d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:14.859126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9358564-ba46-4dda-8011-4a8d5db0a9d6 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:14.859424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9358564-ba46-4dda-8011-4a8d5db0a9d6 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:14.859757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9358564-ba46-4dda-8011-4a8d5db0a9d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:14.861625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c9358564-ba46-4dda-8011-4a8d5db0a9d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:14.959505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9358564-ba46-4dda-8011-4a8d5db0a9d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:14.961386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9358564-ba46-4dda-8011-4a8d5db0a9d6 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:14.983026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 681/1368] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:53:15.357279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25e627-2ba1-426f-bff2-a20fe70bead1 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:15.357851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25e627-2ba1-426f-bff2-a20fe70bead1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:15.357941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25e627-2ba1-426f-bff2-a20fe70bead1 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:15.359554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 688/1369] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:15.365171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f064-18d9-4b2b-be18-4f4e5a7d0dfe None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:15.365510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f064-18d9-4b2b-be18-4f4e5a7d0dfe None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:15.365710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f064-18d9-4b2b-be18-4f4e5a7d0dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:15.403473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:15.450995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5cc0f064-18d9-4b2b-be18-4f4e5a7d0dfe None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:15.453999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5cc0f064-18d9-4b2b-be18-4f4e5a7d0dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:15.568233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 682/1370] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:16.158111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fce41-fa96-47ab-9b81-e520607f9f5f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:16.158111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fce41-fa96-47ab-9b81-e520607f9f5f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:16.158111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fce41-fa96-47ab-9b81-e520607f9f5f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:16.159819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 689/1371] 10.222.0.61 () {54 vars in 964 bytes} [Thu Sep 1 13:53:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:16.164943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895aeb-0760-48ad-9efc-f1dac59588cc None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:16.165413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895aeb-0760-48ad-9efc-f1dac59588cc None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:16.165680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895aeb-0760-48ad-9efc-f1dac59588cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:16.256149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-71895aeb-0760-48ad-9efc-f1dac59588cc None None] MFA Rules not processed for user `caa058ab479540848d2d9793fdfe4e09`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:16.258037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-71895aeb-0760-48ad-9efc-f1dac59588cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:16.324243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:16.361673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 683/1372] 10.222.0.61 () {58 vars in 1061 bytes} [Thu Sep 1 13:53:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:16.372038 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fce41-fa96-47ab-9b81-e520607f9f5f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:16.374462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-196fce41-fa96-47ab-9b81-e520607f9f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:16.498644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fce41-fa96-47ab-9b81-e520607f9f5f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:16.542711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:16.580005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189fc92-b5dc-4832-a822-2fbb50a95e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:16.580621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189fc92-b5dc-4832-a822-2fbb50a95e43 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:16.580898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189fc92-b5dc-4832-a822-2fbb50a95e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:16.581170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189fc92-b5dc-4832-a822-2fbb50a95e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:16.582955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f189fc92-b5dc-4832-a822-2fbb50a95e43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:16.703543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189fc92-b5dc-4832-a822-2fbb50a95e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:16.705159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189fc92-b5dc-4832-a822-2fbb50a95e43 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:16.725675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 690/1373] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:17.239741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe415f-5938-41a4-9843-a72f64f12001 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:17.239741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe415f-5938-41a4-9843-a72f64f12001 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:17.239741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe415f-5938-41a4-9843-a72f64f12001 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:17.241012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 684/1374] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:17.247573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703cc13-82fc-453c-b549-d8b5971c6f51 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:17.247859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703cc13-82fc-453c-b549-d8b5971c6f51 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:17.248130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703cc13-82fc-453c-b549-d8b5971c6f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:17.326085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3703cc13-82fc-453c-b549-d8b5971c6f51 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:17.327315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3703cc13-82fc-453c-b549-d8b5971c6f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:17.350082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:17.436761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 691/1375] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:17.730314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe415f-5938-41a4-9843-a72f64f12001 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:17.731858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1abe415f-5938-41a4-9843-a72f64f12001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:17.880340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe415f-5938-41a4-9843-a72f64f12001 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:17.949618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:18.023116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071561a-7755-43db-a3e4-c8a54336648a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:18.023658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071561a-7755-43db-a3e4-c8a54336648a service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:18.023926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071561a-7755-43db-a3e4-c8a54336648a service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:18.024196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071561a-7755-43db-a3e4-c8a54336648a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:18.025919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3071561a-7755-43db-a3e4-c8a54336648a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:18.121957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071561a-7755-43db-a3e4-c8a54336648a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:18.123863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071561a-7755-43db-a3e4-c8a54336648a service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:18.142318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 685/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:20.264326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfa1fc-e1c0-4143-8879-4ec87114acc5 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:20.265058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfa1fc-e1c0-4143-8879-4ec87114acc5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:20.265340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfa1fc-e1c0-4143-8879-4ec87114acc5 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:20.267226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 692/1377] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:20.273886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791db39-fbfc-4807-a94c-0e73b7c46407 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:20.274350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791db39-fbfc-4807-a94c-0e73b7c46407 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:20.274452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791db39-fbfc-4807-a94c-0e73b7c46407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:20.363629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f791db39-fbfc-4807-a94c-0e73b7c46407 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:20.365086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f791db39-fbfc-4807-a94c-0e73b7c46407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:20.376990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:20.508403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 686/1378] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:20.806360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfa1fc-e1c0-4143-8879-4ec87114acc5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:20.807926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c0cfa1fc-e1c0-4143-8879-4ec87114acc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:20.974174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfa1fc-e1c0-4143-8879-4ec87114acc5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:20.978322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:21.099495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f91216-07b3-4d42-b7ef-b376be2121ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:21.100107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f91216-07b3-4d42-b7ef-b376be2121ce service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:21.100387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f91216-07b3-4d42-b7ef-b376be2121ce service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:21.100659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f91216-07b3-4d42-b7ef-b376be2121ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:21.102478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e8f91216-07b3-4d42-b7ef-b376be2121ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:21.205539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f91216-07b3-4d42-b7ef-b376be2121ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:21.207172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f91216-07b3-4d42-b7ef-b376be2121ce service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:21.226661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 693/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:23.359521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681cfb5-f353-4a5c-a3f2-5dbd05a75232 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:23.360285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681cfb5-f353-4a5c-a3f2-5dbd05a75232 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:23.360285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681cfb5-f353-4a5c-a3f2-5dbd05a75232 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:23.361627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 687/1380] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:23.367428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f001cc-f060-42d2-b93f-f5810859f6e8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:23.367988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f001cc-f060-42d2-b93f-f5810859f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:23.367988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f001cc-f060-42d2-b93f-f5810859f6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:23.391605 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:23.501177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-01f001cc-f060-42d2-b93f-f5810859f6e8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:23.502575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-01f001cc-f060-42d2-b93f-f5810859f6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:23.614181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 694/1381] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:23.915158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681cfb5-f353-4a5c-a3f2-5dbd05a75232 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:23.916747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6681cfb5-f353-4a5c-a3f2-5dbd05a75232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:24.078592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:24.089596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681cfb5-f353-4a5c-a3f2-5dbd05a75232 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:24.208165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123e37a-d3d2-4db0-ba36-bb1d261769b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:24.208774 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123e37a-d3d2-4db0-ba36-bb1d261769b4 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:24.209074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123e37a-d3d2-4db0-ba36-bb1d261769b4 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:24.209338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123e37a-d3d2-4db0-ba36-bb1d261769b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:24.210943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7123e37a-d3d2-4db0-ba36-bb1d261769b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:24.314069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123e37a-d3d2-4db0-ba36-bb1d261769b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:24.315793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123e37a-d3d2-4db0-ba36-bb1d261769b4 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:24.328040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:24.336896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 688/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:26.627012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb4d05-ba34-408f-b177-90fb4eabbdd9 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:26.627700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb4d05-ba34-408f-b177-90fb4eabbdd9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:26.628405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb4d05-ba34-408f-b177-90fb4eabbdd9 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:26.632237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 695/1383] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:26.638449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e886f-0438-464f-a1c4-a5f284881997 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:26.638946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e886f-0438-464f-a1c4-a5f284881997 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:26.639045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e886f-0438-464f-a1c4-a5f284881997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:26.784117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e75e886f-0438-464f-a1c4-a5f284881997 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:26.788135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e75e886f-0438-464f-a1c4-a5f284881997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:26.907657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 689/1384] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:27.199475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb4d05-ba34-408f-b177-90fb4eabbdd9 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:27.201061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9eeb4d05-ba34-408f-b177-90fb4eabbdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:27.310079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:27.375216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb4d05-ba34-408f-b177-90fb4eabbdd9 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:27.516518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b2d35-14fb-45c9-9e5a-3c1cf372782c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:27.517154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b2d35-14fb-45c9-9e5a-3c1cf372782c service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:27.517463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b2d35-14fb-45c9-9e5a-3c1cf372782c service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:27.517713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b2d35-14fb-45c9-9e5a-3c1cf372782c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:27.519502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-368b2d35-14fb-45c9-9e5a-3c1cf372782c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:27.575001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:27.623242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b2d35-14fb-45c9-9e5a-3c1cf372782c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:27.624899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b2d35-14fb-45c9-9e5a-3c1cf372782c service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:27.647170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 696/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:29.659882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c632c-55ea-42b2-ba58-4cb741c4b222 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:29.660527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c632c-55ea-42b2-ba58-4cb741c4b222 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:29.660527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c632c-55ea-42b2-ba58-4cb741c4b222 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:29.662056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 690/1386] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:29.668020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae087191-54a2-41c8-9f03-6693e4c07703 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:29.668582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae087191-54a2-41c8-9f03-6693e4c07703 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:29.668693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae087191-54a2-41c8-9f03-6693e4c07703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:29.754667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ae087191-54a2-41c8-9f03-6693e4c07703 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:29.757735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ae087191-54a2-41c8-9f03-6693e4c07703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:29.908913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 697/1387] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:30.207523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c632c-55ea-42b2-ba58-4cb741c4b222 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:30.209065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-929c632c-55ea-42b2-ba58-4cb741c4b222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:30.299024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:30.375914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c632c-55ea-42b2-ba58-4cb741c4b222 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:30.507922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b404d-cabc-4869-8fea-613f0f66b282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:30.508454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b404d-cabc-4869-8fea-613f0f66b282 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:30.508730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b404d-cabc-4869-8fea-613f0f66b282 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:30.508997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b404d-cabc-4869-8fea-613f0f66b282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:30.511214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c72b404d-cabc-4869-8fea-613f0f66b282 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:30.558211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:30.615948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b404d-cabc-4869-8fea-613f0f66b282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:30.617432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b404d-cabc-4869-8fea-613f0f66b282 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:30.641301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 691/1388] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:53:32.111675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc56a5-986d-4050-ac39-b16597b34a6b None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:32.112222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc56a5-986d-4050-ac39-b16597b34a6b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:32.112222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc56a5-986d-4050-ac39-b16597b34a6b None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:32.113752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 698/1389] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:32.119428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c1acc-662b-46a9-8a58-3cfeb24c4c1e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:32.119869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c1acc-662b-46a9-8a58-3cfeb24c4c1e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:32.119997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c1acc-662b-46a9-8a58-3cfeb24c4c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:32.188201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e24c1acc-662b-46a9-8a58-3cfeb24c4c1e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:32.189314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e24c1acc-662b-46a9-8a58-3cfeb24c4c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:32.327150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 692/1390] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:32.646538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851f48b-2587-4861-b7a0-3ee74f20a718 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:32.647059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851f48b-2587-4861-b7a0-3ee74f20a718 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:32.647444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851f48b-2587-4861-b7a0-3ee74f20a718 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:32.651882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 699/1391] 10.222.0.61 () {54 vars in 967 bytes} [Thu Sep 1 13:53:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:32.658594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f7894-c7a1-46fe-a2a8-217ad284d478 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:32.658877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f7894-c7a1-46fe-a2a8-217ad284d478 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:32.659158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f7894-c7a1-46fe-a2a8-217ad284d478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:32.676107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:32.743171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-8a2f7894-c7a1-46fe-a2a8-217ad284d478 None None] MFA Rules not processed for user `42f33454f4b94b7fb54e83252ce564db`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:32.744617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8a2f7894-c7a1-46fe-a2a8-217ad284d478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:32.801153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bc882-fc4c-4447-afd7-695db609348f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:32.801434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bc882-fc4c-4447-afd7-695db609348f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:32.801714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bc882-fc4c-4447-afd7-695db609348f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:32.803099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 700/1392] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:53:32.808482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:32.808767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:32.809039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:32.831147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 693/1393] 10.222.0.61 () {58 vars in 1064 bytes} [Thu Sep 1 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:32.841149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f7894-c7a1-46fe-a2a8-217ad284d478 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:32.851100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f7894-c7a1-46fe-a2a8-217ad284d478 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:32.852839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:32.895466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:32.896820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:32.934998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781dd49-96cd-482b-a840-b5c0cbe756b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:32.935618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781dd49-96cd-482b-a840-b5c0cbe756b2 service glance] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:32.935900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781dd49-96cd-482b-a840-b5c0cbe756b2 service glance] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:32.936176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781dd49-96cd-482b-a840-b5c0cbe756b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:32.937945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8781dd49-96cd-482b-a840-b5c0cbe756b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:32.972553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:33.022254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 701/1394] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:53:33.074720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781dd49-96cd-482b-a840-b5c0cbe756b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:33.076234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781dd49-96cd-482b-a840-b5c0cbe756b2 service glance] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:33.095457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 694/1395] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:53:33.319091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:33.320414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:33.444280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:33.451939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21e5c4-b406-4d43-810e-534c57ebeed7 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:33.580803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f958b4-164f-4adc-a6a3-2d177ae0c0f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:33.581406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f958b4-164f-4adc-a6a3-2d177ae0c0f4 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:33.581678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f958b4-164f-4adc-a6a3-2d177ae0c0f4 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:33.581973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f958b4-164f-4adc-a6a3-2d177ae0c0f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:33.583547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-82f958b4-164f-4adc-a6a3-2d177ae0c0f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:33.678202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f958b4-164f-4adc-a6a3-2d177ae0c0f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:33.679804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f958b4-164f-4adc-a6a3-2d177ae0c0f4 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:33.696977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:33.700359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 702/1396] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:34.004635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7f6a2-b430-4353-a4c8-a465bb442df2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:34.005206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7f6a2-b430-4353-a4c8-a465bb442df2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:34.005444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7f6a2-b430-4353-a4c8-a465bb442df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:34.081880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-74e7f6a2-b430-4353-a4c8-a465bb442df2 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:34.083050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-74e7f6a2-b430-4353-a4c8-a465bb442df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:34.167446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:34.173753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 695/1397] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 13:53:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:34.222657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37b982-8b38-45b2-b88d-276f5d6e7052 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:34.222943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37b982-8b38-45b2-b88d-276f5d6e7052 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:34.223216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37b982-8b38-45b2-b88d-276f5d6e7052 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:34.224579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 703/1398] 10.222.0.61 () {54 vars in 943 bytes} [Thu Sep 1 13:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:34.231529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035d89b-fe16-465e-8eb1-561b529c20db None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:34.231826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035d89b-fe16-465e-8eb1-561b529c20db None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:34.232100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035d89b-fe16-465e-8eb1-561b529c20db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:34.314771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2035d89b-fe16-465e-8eb1-561b529c20db None None] MFA Rules not processed for user `0f29e56690b343269681dfd8caffe001`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:34.315905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2035d89b-fe16-465e-8eb1-561b529c20db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:34.427970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 696/1399] 10.222.0.61 () {58 vars in 1040 bytes} [Thu Sep 1 13:53:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:34.438356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37b982-8b38-45b2-b88d-276f5d6e7052 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:34.439881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7f37b982-8b38-45b2-b88d-276f5d6e7052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:34.577720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37b982-8b38-45b2-b88d-276f5d6e7052 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:34.672879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:34.694043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8654d-04fc-4f06-ae87-d5531919af97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:34.694571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8654d-04fc-4f06-ae87-d5531919af97 service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:34.694847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8654d-04fc-4f06-ae87-d5531919af97 service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:34.695117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8654d-04fc-4f06-ae87-d5531919af97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:34.697341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c5b8654d-04fc-4f06-ae87-d5531919af97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:34.791838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8654d-04fc-4f06-ae87-d5531919af97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:34.793465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8654d-04fc-4f06-ae87-d5531919af97 service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:34.804124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 704/1400] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 13:53:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 13:53:35.875130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bad8b4-75a5-4997-b813-0a1ed931d88d None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:35.875713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bad8b4-75a5-4997-b813-0a1ed931d88d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:35.875713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bad8b4-75a5-4997-b813-0a1ed931d88d None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:35.877042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 697/1401] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:35.882399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384a08-14b2-4263-a0f8-9a5a7fcd6545 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:35.882889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384a08-14b2-4263-a0f8-9a5a7fcd6545 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:35.882972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384a08-14b2-4263-a0f8-9a5a7fcd6545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:35.982984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-fe384a08-14b2-4263-a0f8-9a5a7fcd6545 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:35.984335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fe384a08-14b2-4263-a0f8-9a5a7fcd6545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:36.075764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:36.104682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 705/1402] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:36.395142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bad8b4-75a5-4997-b813-0a1ed931d88d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:36.396668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-46bad8b4-75a5-4997-b813-0a1ed931d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:36.465072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:36.570669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bad8b4-75a5-4997-b813-0a1ed931d88d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:36.709295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af610d5e-18a1-480b-a372-09e657702302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:36.709948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af610d5e-18a1-480b-a372-09e657702302 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:36.710228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af610d5e-18a1-480b-a372-09e657702302 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:36.710491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af610d5e-18a1-480b-a372-09e657702302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:36.712166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-af610d5e-18a1-480b-a372-09e657702302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:36.729733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:36.819931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af610d5e-18a1-480b-a372-09e657702302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:36.822179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af610d5e-18a1-480b-a372-09e657702302 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:36.844656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 698/1403] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:38.979697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483423-d9f7-4f16-aae1-c2e903d02e13 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:38.980758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483423-d9f7-4f16-aae1-c2e903d02e13 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:38.981043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483423-d9f7-4f16-aae1-c2e903d02e13 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:38.982452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 706/1404] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:38.988085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863387-bebb-4749-94f5-ccfd26adb50c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:38.988766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863387-bebb-4749-94f5-ccfd26adb50c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:38.988766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863387-bebb-4749-94f5-ccfd26adb50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:39.076888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-40863387-bebb-4749-94f5-ccfd26adb50c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:39.078536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-40863387-bebb-4749-94f5-ccfd26adb50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:39.202625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:39.213528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 699/1405] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:39.519228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47483423-d9f7-4f16-aae1-c2e903d02e13 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:39.520750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-47483423-d9f7-4f16-aae1-c2e903d02e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:39.691096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47483423-d9f7-4f16-aae1-c2e903d02e13 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:39.753518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:39.822450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa8f96-7e9a-4d0c-b29c-2fffbf1e6cce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:39.823124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa8f96-7e9a-4d0c-b29c-2fffbf1e6cce service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:39.823398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa8f96-7e9a-4d0c-b29c-2fffbf1e6cce service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:39.823667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa8f96-7e9a-4d0c-b29c-2fffbf1e6cce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:39.825330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4caa8f96-7e9a-4d0c-b29c-2fffbf1e6cce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:39.930509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa8f96-7e9a-4d0c-b29c-2fffbf1e6cce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:39.932284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa8f96-7e9a-4d0c-b29c-2fffbf1e6cce service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:39.956975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 707/1406] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:42.201590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ca49-7261-4b21-b768-82079b215019 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:42.202175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ca49-7261-4b21-b768-82079b215019 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:42.202175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ca49-7261-4b21-b768-82079b215019 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:42.203457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 700/1407] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:42.209417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88a0e-86ff-4dcc-9450-7777a3bee09e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:42.210194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88a0e-86ff-4dcc-9450-7777a3bee09e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:42.210194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88a0e-86ff-4dcc-9450-7777a3bee09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:42.300293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c8e88a0e-86ff-4dcc-9450-7777a3bee09e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:42.301524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c8e88a0e-86ff-4dcc-9450-7777a3bee09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:42.320058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:42.427040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 708/1408] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:53:42.713237 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ca49-7261-4b21-b768-82079b215019 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:53:42.714808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4f22ca49-7261-4b21-b768-82079b215019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:42.880472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ca49-7261-4b21-b768-82079b215019 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:53:42.956812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:43.013276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e83eb-8c5f-4d7d-9895-6a921b069351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:53:43.014014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e83eb-8c5f-4d7d-9895-6a921b069351 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:43.014296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e83eb-8c5f-4d7d-9895-6a921b069351 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:43.014621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e83eb-8c5f-4d7d-9895-6a921b069351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:43.016488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f34e83eb-8c5f-4d7d-9895-6a921b069351 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:43.123871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e83eb-8c5f-4d7d-9895-6a921b069351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:53:43.125433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e83eb-8c5f-4d7d-9895-6a921b069351 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:53:43.147979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 701/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:53:52.417283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=37 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:53.144175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=65 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:58.188189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb399c-ac66-4d4f-ae0c-41d9cfd54ba1 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:58.188733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb399c-ac66-4d4f-ae0c-41d9cfd54ba1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:58.188733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb399c-ac66-4d4f-ae0c-41d9cfd54ba1 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:58.190053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 709/1410] 10.222.0.61 () {54 vars in 918 bytes} [Thu Sep 1 13:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:58.195373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab7e65-81e2-4ff5-b735-b66733685bdc None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:58.195792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab7e65-81e2-4ff5-b735-b66733685bdc None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:58.195952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab7e65-81e2-4ff5-b735-b66733685bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:58.271407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-edab7e65-81e2-4ff5-b735-b66733685bdc None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:58.272608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-edab7e65-81e2-4ff5-b735-b66733685bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:58.379856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 702/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Thu Sep 1 13:53:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:53:58.395257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8232b1-5598-45ff-b868-58ab09a00818 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:58.395586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8232b1-5598-45ff-b868-58ab09a00818 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:58.395836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8232b1-5598-45ff-b868-58ab09a00818 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:58.397177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 710/1412] 10.222.0.61 () {54 vars in 918 bytes} [Thu Sep 1 13:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:53:58.402489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964bc7-f456-4e20-a662-e52db8030534 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:53:58.402769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964bc7-f456-4e20-a662-e52db8030534 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:53:58.403040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964bc7-f456-4e20-a662-e52db8030534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:53:58.478540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:53:58.481206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-78964bc7-f456-4e20-a662-e52db8030534 None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:53:58.482478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-78964bc7-f456-4e20-a662-e52db8030534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:53:58.600134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 703/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Thu Sep 1 13:53:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:01.013029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4140028-d23a-44b9-8a92-b0f35cb8a34e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:01.013550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4140028-d23a-44b9-8a92-b0f35cb8a34e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:01.013550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4140028-d23a-44b9-8a92-b0f35cb8a34e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:01.015211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:01.020900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18582fb-021c-4b38-90b1-64575522a7c0 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:01.021348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18582fb-021c-4b38-90b1-64575522a7c0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:01.021430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18582fb-021c-4b38-90b1-64575522a7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:01.091853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c18582fb-021c-4b38-90b1-64575522a7c0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:01.093080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c18582fb-021c-4b38-90b1-64575522a7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:01.149723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:01.198011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:01.495100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4140028-d23a-44b9-8a92-b0f35cb8a34e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:01.497034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c4140028-d23a-44b9-8a92-b0f35cb8a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:01.644159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4140028-d23a-44b9-8a92-b0f35cb8a34e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:01.717441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:01.757944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d533be9-20f4-4c06-8621-386fda0628ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:01.758615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d533be9-20f4-4c06-8621-386fda0628ea service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:01.758824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d533be9-20f4-4c06-8621-386fda0628ea service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:01.759093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d533be9-20f4-4c06-8621-386fda0628ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:01.760687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5d533be9-20f4-4c06-8621-386fda0628ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:01.851082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d533be9-20f4-4c06-8621-386fda0628ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:01.852739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d533be9-20f4-4c06-8621-386fda0628ea service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:01.876040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 712/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:02.777373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807cba-212f-4f20-9e80-9d9931107e39 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:02.778033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807cba-212f-4f20-9e80-9d9931107e39 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:02.778033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807cba-212f-4f20-9e80-9d9931107e39 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:02.779364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:02.784764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25a184-4a20-4042-9281-f7e6e209abc4 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:02.785249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25a184-4a20-4042-9281-f7e6e209abc4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:02.785354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25a184-4a20-4042-9281-f7e6e209abc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:02.863955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-5c25a184-4a20-4042-9281-f7e6e209abc4 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:02.865202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5c25a184-4a20-4042-9281-f7e6e209abc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:02.893169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:02.965489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:03.118606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55807cba-212f-4f20-9e80-9d9931107e39 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:03.120110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-55807cba-212f-4f20-9e80-9d9931107e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:03.264335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55807cba-212f-4f20-9e80-9d9931107e39 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:03.306864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:03.384566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e842bb-b7bc-4374-9513-920abf10a187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:03.385084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e842bb-b7bc-4374-9513-920abf10a187 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:03.385357 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e842bb-b7bc-4374-9513-920abf10a187 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:03.385621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e842bb-b7bc-4374-9513-920abf10a187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:03.387310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d5e842bb-b7bc-4374-9513-920abf10a187 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:03.480537 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e842bb-b7bc-4374-9513-920abf10a187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:03.483290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e842bb-b7bc-4374-9513-920abf10a187 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:03.510084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 706/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:04.419692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc51d-d7e5-48a4-9bec-b6761723f6a7 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:04.420587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc51d-d7e5-48a4-9bec-b6761723f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:04.420587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfc51d-d7e5-48a4-9bec-b6761723f6a7 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:04.422201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:54:04.428365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f651-348d-464f-9faa-89840900148c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:04.428949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f651-348d-464f-9faa-89840900148c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:04.428949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f651-348d-464f-9faa-89840900148c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:04.491858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:04.518750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2291f651-348d-464f-9faa-89840900148c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:04.520048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2291f651-348d-464f-9faa-89840900148c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:04.655605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:04.815484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfc51d-d7e5-48a4-9bec-b6761723f6a7 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:04.817088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-47cfc51d-d7e5-48a4-9bec-b6761723f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:05.033138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfc51d-d7e5-48a4-9bec-b6761723f6a7 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:05.047470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:05.158989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110a9e5-7885-493f-82a3-d18c00608676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:05.159605 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110a9e5-7885-493f-82a3-d18c00608676 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:05.159883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110a9e5-7885-493f-82a3-d18c00608676 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:05.160150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110a9e5-7885-493f-82a3-d18c00608676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:05.161831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c110a9e5-7885-493f-82a3-d18c00608676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:05.265122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110a9e5-7885-493f-82a3-d18c00608676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:05.267971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110a9e5-7885-493f-82a3-d18c00608676 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:05.303141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 715/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:54:06.783972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe62661-b99b-4eb9-9fdf-36f88732c9a1 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:06.784577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe62661-b99b-4eb9-9fdf-36f88732c9a1 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:06.784577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe62661-b99b-4eb9-9fdf-36f88732c9a1 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:06.785910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:06.791420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b2d5-5572-422c-98fa-4ce634efe638 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:06.791991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b2d5-5572-422c-98fa-4ce634efe638 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:06.792247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b2d5-5572-422c-98fa-4ce634efe638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:06.813963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:06.878072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c4f7b2d5-5572-422c-98fa-4ce634efe638 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:06.879311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c4f7b2d5-5572-422c-98fa-4ce634efe638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:06.991113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:07.142546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe62661-b99b-4eb9-9fdf-36f88732c9a1 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:07.144101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fbe62661-b99b-4eb9-9fdf-36f88732c9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:07.268819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:07.313808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe62661-b99b-4eb9-9fdf-36f88732c9a1 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:07.466137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b46120-12f3-47ed-b2b2-9a62986211ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:07.466956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46120-12f3-47ed-b2b2-9a62986211ce service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:07.467298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46120-12f3-47ed-b2b2-9a62986211ce service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:07.467616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46120-12f3-47ed-b2b2-9a62986211ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:07.469345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-59b46120-12f3-47ed-b2b2-9a62986211ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:07.557762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:07.573243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b46120-12f3-47ed-b2b2-9a62986211ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:07.574904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b46120-12f3-47ed-b2b2-9a62986211ce service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:07.596342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 709/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:08.508127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43f34-31fb-4aca-8ee2-baa9014f930f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:08.508662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43f34-31fb-4aca-8ee2-baa9014f930f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:08.508757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43f34-31fb-4aca-8ee2-baa9014f930f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:08.510278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:08.519511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811d3f4-0f35-4aca-becb-490754597cb0 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:08.519511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811d3f4-0f35-4aca-becb-490754597cb0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:08.519511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811d3f4-0f35-4aca-becb-490754597cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:08.605489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f811d3f4-0f35-4aca-becb-490754597cb0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:08.606846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f811d3f4-0f35-4aca-becb-490754597cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:08.716035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:08.872426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb43f34-31fb-4aca-8ee2-baa9014f930f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:08.873981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0eb43f34-31fb-4aca-8ee2-baa9014f930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:08.963374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:09.035330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb43f34-31fb-4aca-8ee2-baa9014f930f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:09.159893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207ad1e-c4f4-48ff-984d-e178dd3448ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:09.160417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ad1e-c4f4-48ff-984d-e178dd3448ac service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:09.160697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ad1e-c4f4-48ff-984d-e178dd3448ac service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:09.160972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ad1e-c4f4-48ff-984d-e178dd3448ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:09.162783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9207ad1e-c4f4-48ff-984d-e178dd3448ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:09.214313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:09.265453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207ad1e-c4f4-48ff-984d-e178dd3448ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:09.267302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207ad1e-c4f4-48ff-984d-e178dd3448ac service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:09.287471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 718/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:10.207467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eceb7-a3a8-4711-a916-642346d3e9c2 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:10.208151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eceb7-a3a8-4711-a916-642346d3e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:10.208151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eceb7-a3a8-4711-a916-642346d3e9c2 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:10.209253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:54:10.214923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687c8a-6531-4ca0-bb19-5386069c72c9 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:10.215781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687c8a-6531-4ca0-bb19-5386069c72c9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:10.215781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687c8a-6531-4ca0-bb19-5386069c72c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:10.302518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-25687c8a-6531-4ca0-bb19-5386069c72c9 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:10.303745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-25687c8a-6531-4ca0-bb19-5386069c72c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:10.411090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:10.565502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eceb7-a3a8-4711-a916-642346d3e9c2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:10.566993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9c0eceb7-a3a8-4711-a916-642346d3e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:10.621086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:10.742380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eceb7-a3a8-4711-a916-642346d3e9c2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:10.878006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617874f-98b3-4e49-9bf3-ff6c164e99c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:10.879039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617874f-98b3-4e49-9bf3-ff6c164e99c5 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:10.879541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617874f-98b3-4e49-9bf3-ff6c164e99c5 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:10.880052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617874f-98b3-4e49-9bf3-ff6c164e99c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:10.883460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a617874f-98b3-4e49-9bf3-ff6c164e99c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:10.896619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:11.003208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617874f-98b3-4e49-9bf3-ff6c164e99c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:11.004855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617874f-98b3-4e49-9bf3-ff6c164e99c5 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:11.034079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 712/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:54:11.967829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e990b-5f9a-4eb4-9592-95856409c41a None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:11.968441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e990b-5f9a-4eb4-9592-95856409c41a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:11.968441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e990b-5f9a-4eb4-9592-95856409c41a None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:11.969795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:54:11.975582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fc90b-02d1-47b9-a195-b653d781eeb3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:11.976217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fc90b-02d1-47b9-a195-b653d781eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:11.976856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fc90b-02d1-47b9-a195-b653d781eeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:12.065235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2e6fc90b-02d1-47b9-a195-b653d781eeb3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:12.066685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2e6fc90b-02d1-47b9-a195-b653d781eeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:12.141076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:12.174693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:12.335018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e990b-5f9a-4eb4-9592-95856409c41a None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:12.337223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7b0e990b-5f9a-4eb4-9592-95856409c41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:12.357111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:12.523888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e990b-5f9a-4eb4-9592-95856409c41a None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:12.628481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:12.660437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25426b-ee69-45da-a3ff-902a0dc32b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:12.661059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25426b-ee69-45da-a3ff-902a0dc32b35 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:12.661341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25426b-ee69-45da-a3ff-902a0dc32b35 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:12.661614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25426b-ee69-45da-a3ff-902a0dc32b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:12.663429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4d25426b-ee69-45da-a3ff-902a0dc32b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:12.770175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25426b-ee69-45da-a3ff-902a0dc32b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:12.772001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25426b-ee69-45da-a3ff-902a0dc32b35 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:12.794905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 721/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:54:13.762654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56ccc-b951-4460-8609-a126b0db7e6f None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:13.763380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56ccc-b951-4460-8609-a126b0db7e6f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:13.763380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56ccc-b951-4460-8609-a126b0db7e6f None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:13.765041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:54:13.770957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae230-b3bd-40d5-806c-3d4637fc9426 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:13.771558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae230-b3bd-40d5-806c-3d4637fc9426 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:13.771558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae230-b3bd-40d5-806c-3d4637fc9426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:13.862944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-996ae230-b3bd-40d5-806c-3d4637fc9426 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:13.864197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-996ae230-b3bd-40d5-806c-3d4637fc9426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:13.917169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:13.986479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:14.143397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d56ccc-b951-4460-8609-a126b0db7e6f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:14.145006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-20d56ccc-b951-4460-8609-a126b0db7e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:14.336076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d56ccc-b951-4460-8609-a126b0db7e6f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:14.405875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:14.475776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fd771-4000-43bb-8ca6-b2e18b92559b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:14.476957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fd771-4000-43bb-8ca6-b2e18b92559b service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:14.477225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fd771-4000-43bb-8ca6-b2e18b92559b service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:14.477492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fd771-4000-43bb-8ca6-b2e18b92559b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:14.479307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5a3fd771-4000-43bb-8ca6-b2e18b92559b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:14.638473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fd771-4000-43bb-8ca6-b2e18b92559b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:14.641325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fd771-4000-43bb-8ca6-b2e18b92559b service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:14.674234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 715/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:54:15.644722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab04e9-87be-465f-855b-a1947c6a4789 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:15.645316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab04e9-87be-465f-855b-a1947c6a4789 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:15.645661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab04e9-87be-465f-855b-a1947c6a4789 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:15.648219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:54:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:15.654474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b055cc-a2f9-4098-a43c-446507579255 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:15.655064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b055cc-a2f9-4098-a43c-446507579255 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:15.655064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b055cc-a2f9-4098-a43c-446507579255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:15.763105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-44b055cc-a2f9-4098-a43c-446507579255 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:15.764369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-44b055cc-a2f9-4098-a43c-446507579255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:15.766797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:15.886833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:16.044028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab04e9-87be-465f-855b-a1947c6a4789 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:16.045668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-acab04e9-87be-465f-855b-a1947c6a4789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:16.229887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab04e9-87be-465f-855b-a1947c6a4789 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:16.262257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:16.410655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69814904-7838-4012-8ace-fa68839d0bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:16.411670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814904-7838-4012-8ace-fa68839d0bed service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:16.412163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814904-7838-4012-8ace-fa68839d0bed service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:16.412646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814904-7838-4012-8ace-fa68839d0bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:16.416133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-69814904-7838-4012-8ace-fa68839d0bed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:16.551776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69814904-7838-4012-8ace-fa68839d0bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:16.553471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69814904-7838-4012-8ace-fa68839d0bed service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:16.575954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 724/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:18.519769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd4eb-c59d-41aa-afdd-9725386e588a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:18.520405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd4eb-c59d-41aa-afdd-9725386e588a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:18.520513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd4eb-c59d-41aa-afdd-9725386e588a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:18.522005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:54:18.529131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e204663-5f0e-4b29-a7c2-53018f956bc8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:18.529752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e204663-5f0e-4b29-a7c2-53018f956bc8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:18.529752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e204663-5f0e-4b29-a7c2-53018f956bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:18.579055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:18.610867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9e204663-5f0e-4b29-a7c2-53018f956bc8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:18.612969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9e204663-5f0e-4b29-a7c2-53018f956bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:18.857213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:19.154909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd4eb-c59d-41aa-afdd-9725386e588a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:19.156570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-93bfd4eb-c59d-41aa-afdd-9725386e588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:19.285880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd4eb-c59d-41aa-afdd-9725386e588a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:19.327194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:19.377918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25433eb-9940-4db5-8c23-11c7c7497f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:19.378592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25433eb-9940-4db5-8c23-11c7c7497f75 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:19.378927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25433eb-9940-4db5-8c23-11c7c7497f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:19.379232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25433eb-9940-4db5-8c23-11c7c7497f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:19.381051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c25433eb-9940-4db5-8c23-11c7c7497f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:19.516604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25433eb-9940-4db5-8c23-11c7c7497f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:19.518100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25433eb-9940-4db5-8c23-11c7c7497f75 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:19.542201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 718/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:54:21.749325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947564e-b5f7-4534-8ee6-cc429c747f60 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:21.750019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947564e-b5f7-4534-8ee6-cc429c747f60 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:21.750019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947564e-b5f7-4534-8ee6-cc429c747f60 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:21.751305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:54:21.758269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd497fc-2b52-4585-83be-33273cecdc8a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:21.759094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd497fc-2b52-4585-83be-33273cecdc8a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:21.759310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd497fc-2b52-4585-83be-33273cecdc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:21.841144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3fd497fc-2b52-4585-83be-33273cecdc8a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:21.842426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3fd497fc-2b52-4585-83be-33273cecdc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:21.858452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:21.962271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:22.255930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947564e-b5f7-4534-8ee6-cc429c747f60 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:22.257563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c947564e-b5f7-4534-8ee6-cc429c747f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:22.430815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947564e-b5f7-4534-8ee6-cc429c747f60 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:22.451390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:22.527291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8fd7a-fdb0-404d-84a7-53801c52d3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:22.527814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8fd7a-fdb0-404d-84a7-53801c52d3eb service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:22.528091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8fd7a-fdb0-404d-84a7-53801c52d3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:22.528360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8fd7a-fdb0-404d-84a7-53801c52d3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:22.530115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-33f8fd7a-fdb0-404d-84a7-53801c52d3eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:22.660228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8fd7a-fdb0-404d-84a7-53801c52d3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:22.661936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8fd7a-fdb0-404d-84a7-53801c52d3eb service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:22.684349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 727/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:25.169104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22990591-d6b7-4f0b-ad93-d2115fcc9def None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:25.169708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22990591-d6b7-4f0b-ad93-d2115fcc9def None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:25.169708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22990591-d6b7-4f0b-ad93-d2115fcc9def None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:25.171093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:54:25.177649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3801b8f-027c-4938-8428-8b752dd1ee6c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:25.178268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3801b8f-027c-4938-8428-8b752dd1ee6c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:25.178268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3801b8f-027c-4938-8428-8b752dd1ee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:25.256656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:25.283411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e3801b8f-027c-4938-8428-8b752dd1ee6c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:25.284672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e3801b8f-027c-4938-8428-8b752dd1ee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:25.406815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:25.702588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22990591-d6b7-4f0b-ad93-d2115fcc9def None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:25.704374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-22990591-d6b7-4f0b-ad93-d2115fcc9def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:25.829153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22990591-d6b7-4f0b-ad93-d2115fcc9def None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:25.877936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:25.920591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b08fea-fd63-45be-95b0-1cbc2ca38366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:25.921260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b08fea-fd63-45be-95b0-1cbc2ca38366 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:25.921576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b08fea-fd63-45be-95b0-1cbc2ca38366 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:25.921937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b08fea-fd63-45be-95b0-1cbc2ca38366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:25.923705 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-25b08fea-fd63-45be-95b0-1cbc2ca38366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:26.064035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b08fea-fd63-45be-95b0-1cbc2ca38366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:26.065417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b08fea-fd63-45be-95b0-1cbc2ca38366 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:26.089354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 721/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:54:28.290611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669c32c-9981-42c7-bd76-3092cc1818f2 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:28.291456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669c32c-9981-42c7-bd76-3092cc1818f2 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:28.291456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669c32c-9981-42c7-bd76-3092cc1818f2 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:28.292862 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:28.299347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3dcd0-4336-4876-900f-091fbd11a97d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:28.300023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3dcd0-4336-4876-900f-091fbd11a97d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:28.300023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3dcd0-4336-4876-900f-091fbd11a97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:28.392734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-99e3dcd0-4336-4876-900f-091fbd11a97d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:28.394887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-99e3dcd0-4336-4876-900f-091fbd11a97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:28.435139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:28.580562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:28.874486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669c32c-9981-42c7-bd76-3092cc1818f2 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:28.876077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9669c32c-9981-42c7-bd76-3092cc1818f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:28.996392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669c32c-9981-42c7-bd76-3092cc1818f2 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:29.021001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:29.083364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95710f-06d2-47bf-9195-f3bf4c3010e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:29.083990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95710f-06d2-47bf-9195-f3bf4c3010e2 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:29.084269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95710f-06d2-47bf-9195-f3bf4c3010e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:29.084558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95710f-06d2-47bf-9195-f3bf4c3010e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:29.086947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1c95710f-06d2-47bf-9195-f3bf4c3010e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:29.239410 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95710f-06d2-47bf-9195-f3bf4c3010e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:29.241559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95710f-06d2-47bf-9195-f3bf4c3010e2 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:29.264944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 730/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:32.890003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f8fe-bb74-4ccf-9dca-6a7a43ab8c0a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:32.890593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f8fe-bb74-4ccf-9dca-6a7a43ab8c0a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:32.890593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f8fe-bb74-4ccf-9dca-6a7a43ab8c0a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:32.892177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:32.899137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38361e01-bad0-479e-bc46-052d8db2bd6a None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:32.899708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38361e01-bad0-479e-bc46-052d8db2bd6a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:32.899861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38361e01-bad0-479e-bc46-052d8db2bd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:32.978103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:32.982471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-38361e01-bad0-479e-bc46-052d8db2bd6a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:32.984007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-38361e01-bad0-479e-bc46-052d8db2bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:33.108437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:33.503235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f8fe-bb74-4ccf-9dca-6a7a43ab8c0a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:33.504837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cae4f8fe-bb74-4ccf-9dca-6a7a43ab8c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:33.675953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f8fe-bb74-4ccf-9dca-6a7a43ab8c0a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:33.774825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:33.829181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d310c-b066-4ca9-abf1-fb1164ee4868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:33.829880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d310c-b066-4ca9-abf1-fb1164ee4868 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:33.830348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d310c-b066-4ca9-abf1-fb1164ee4868 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:33.830525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d310c-b066-4ca9-abf1-fb1164ee4868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:33.832398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a47d310c-b066-4ca9-abf1-fb1164ee4868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:33.971994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d310c-b066-4ca9-abf1-fb1164ee4868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:33.973487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d310c-b066-4ca9-abf1-fb1164ee4868 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:33.998900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 724/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:36.140095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ab93-e614-46e6-ba3a-1718cf62ac19 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:36.143877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ab93-e614-46e6-ba3a-1718cf62ac19 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:36.143877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ab93-e614-46e6-ba3a-1718cf62ac19 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:36.143877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:36.148686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e908-0521-4abd-9ad2-79ed3783777b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:36.149279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e908-0521-4abd-9ad2-79ed3783777b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:36.149279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e908-0521-4abd-9ad2-79ed3783777b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:36.229751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-0593e908-0521-4abd-9ad2-79ed3783777b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:36.230977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0593e908-0521-4abd-9ad2-79ed3783777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:36.264766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:36.352590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:54:36.646077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38ab93-e614-46e6-ba3a-1718cf62ac19 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:36.648321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6d38ab93-e614-46e6-ba3a-1718cf62ac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:36.767631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38ab93-e614-46e6-ba3a-1718cf62ac19 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:36.799514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:36.853517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c307e5-8930-4cce-9b7c-bdd22344c0c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:36.854163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c307e5-8930-4cce-9b7c-bdd22344c0c1 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:36.854430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c307e5-8930-4cce-9b7c-bdd22344c0c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:36.854691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c307e5-8930-4cce-9b7c-bdd22344c0c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:36.856390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-90c307e5-8930-4cce-9b7c-bdd22344c0c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:36.980757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c307e5-8930-4cce-9b7c-bdd22344c0c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:36.982197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c307e5-8930-4cce-9b7c-bdd22344c0c1 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:37.004517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 733/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:39.113557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af4688-1379-4cbd-8bce-77e970e2f03c None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:39.114371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af4688-1379-4cbd-8bce-77e970e2f03c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:39.114371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af4688-1379-4cbd-8bce-77e970e2f03c None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:39.116080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:39.122368 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebc35-565e-49b0-96cc-cbd1ac34b69c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:39.122938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebc35-565e-49b0-96cc-cbd1ac34b69c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:39.122938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebc35-565e-49b0-96cc-cbd1ac34b69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:39.196071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-436ebc35-565e-49b0-96cc-cbd1ac34b69c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:39.197307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-436ebc35-565e-49b0-96cc-cbd1ac34b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:39.206326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:39.316145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:39.602316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af4688-1379-4cbd-8bce-77e970e2f03c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:39.604940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-79af4688-1379-4cbd-8bce-77e970e2f03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:39.720910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af4688-1379-4cbd-8bce-77e970e2f03c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:39.785308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:39.808094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025ade1-ae4b-4ba8-be4f-b612e4ffbc64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:39.808722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ade1-ae4b-4ba8-be4f-b612e4ffbc64 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:39.809013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ade1-ae4b-4ba8-be4f-b612e4ffbc64 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:39.809259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ade1-ae4b-4ba8-be4f-b612e4ffbc64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:39.811015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9025ade1-ae4b-4ba8-be4f-b612e4ffbc64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:39.948317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025ade1-ae4b-4ba8-be4f-b612e4ffbc64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:39.950613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025ade1-ae4b-4ba8-be4f-b612e4ffbc64 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:39.981218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 727/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:42.139679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30822c9e-c28e-4575-a940-7471adce1a3e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:42.140624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30822c9e-c28e-4575-a940-7471adce1a3e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:42.140624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30822c9e-c28e-4575-a940-7471adce1a3e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:42.141718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:42.147911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e41a3-49b5-46ad-b80e-6dd365f3358b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:42.148624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e41a3-49b5-46ad-b80e-6dd365f3358b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:42.148624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e41a3-49b5-46ad-b80e-6dd365f3358b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:42.233250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b63e41a3-49b5-46ad-b80e-6dd365f3358b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:42.234475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b63e41a3-49b5-46ad-b80e-6dd365f3358b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:42.288471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:42.364298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:42.662256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30822c9e-c28e-4575-a940-7471adce1a3e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:42.666322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-30822c9e-c28e-4575-a940-7471adce1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:42.830519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30822c9e-c28e-4575-a940-7471adce1a3e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:42.876306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:42.917211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d8c7b-16fc-4f81-b260-69a1ec241638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:42.918138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8c7b-16fc-4f81-b260-69a1ec241638 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:42.918637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8c7b-16fc-4f81-b260-69a1ec241638 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:42.919321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8c7b-16fc-4f81-b260-69a1ec241638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:42.921122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-088d8c7b-16fc-4f81-b260-69a1ec241638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:43.058142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d8c7b-16fc-4f81-b260-69a1ec241638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:43.059704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d8c7b-16fc-4f81-b260-69a1ec241638 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:43.081834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 736/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:45.172002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4a24-ffef-4b6a-9465-572c255e4716 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:45.172625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4a24-ffef-4b6a-9465-572c255e4716 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:45.172625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4a24-ffef-4b6a-9465-572c255e4716 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:45.174040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:45.180246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078356ba-a9f0-40bb-a0c9-fd830b2a3d1c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:45.180984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078356ba-a9f0-40bb-a0c9-fd830b2a3d1c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:45.180984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078356ba-a9f0-40bb-a0c9-fd830b2a3d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:45.285939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-078356ba-a9f0-40bb-a0c9-fd830b2a3d1c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:45.287872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-078356ba-a9f0-40bb-a0c9-fd830b2a3d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:45.313280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:45.414835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:45.700542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4a24-ffef-4b6a-9465-572c255e4716 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:45.702135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9f0c4a24-ffef-4b6a-9465-572c255e4716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:45.811149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4a24-ffef-4b6a-9465-572c255e4716 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:45.884253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:45.891382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161449df-481c-4d49-b7b9-f4f920a13e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:45.891998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161449df-481c-4d49-b7b9-f4f920a13e8a service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:45.892285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161449df-481c-4d49-b7b9-f4f920a13e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:45.892554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161449df-481c-4d49-b7b9-f4f920a13e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:45.894347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-161449df-481c-4d49-b7b9-f4f920a13e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:46.035165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161449df-481c-4d49-b7b9-f4f920a13e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:46.037838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161449df-481c-4d49-b7b9-f4f920a13e8a service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:46.062706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 730/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:48.126796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd350e2-b214-40f2-b278-73b1730adf33 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:48.127395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd350e2-b214-40f2-b278-73b1730adf33 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:48.127395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd350e2-b214-40f2-b278-73b1730adf33 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:48.128941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:54:48.135016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab51aa-bbe6-4d60-aa1a-1aecd7cdbfee None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:48.135631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab51aa-bbe6-4d60-aa1a-1aecd7cdbfee None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:48.135631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab51aa-bbe6-4d60-aa1a-1aecd7cdbfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:48.252390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-cfab51aa-bbe6-4d60-aa1a-1aecd7cdbfee None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:54:48.253560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cfab51aa-bbe6-4d60-aa1a-1aecd7cdbfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:48.330688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:48.385878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:54:48.681502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd350e2-b214-40f2-b278-73b1730adf33 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:54:48.683146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fcd350e2-b214-40f2-b278-73b1730adf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:48.797962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd350e2-b214-40f2-b278-73b1730adf33 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:54:48.860615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:48.892938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69f919-5820-4a64-850b-608d8f6a77b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:54:48.893549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69f919-5820-4a64-850b-608d8f6a77b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:54:48.893861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69f919-5820-4a64-850b-608d8f6a77b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:54:48.894129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69f919-5820-4a64-850b-608d8f6a77b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:54:48.895833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3c69f919-5820-4a64-850b-608d8f6a77b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:54:49.020495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69f919-5820-4a64-850b-608d8f6a77b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:54:49.021959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69f919-5820-4a64-850b-608d8f6a77b0 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:54:49.043272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 739/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:54:58.383924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=22 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:54:59.040748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=66 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:10.023549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a77ef-eb62-442e-9d8c-ba2c2b99804a None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:10.024019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a77ef-eb62-442e-9d8c-ba2c2b99804a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:10.024151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a77ef-eb62-442e-9d8c-ba2c2b99804a None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:10.025518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 732/1471] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:55:10.031557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f10bac-dd16-45a0-8fd0-b7fc587f3be0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:10.032097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f10bac-dd16-45a0-8fd0-b7fc587f3be0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:10.032097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f10bac-dd16-45a0-8fd0-b7fc587f3be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:10.118659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c4f10bac-dd16-45a0-8fd0-b7fc587f3be0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:10.119928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c4f10bac-dd16-45a0-8fd0-b7fc587f3be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:10.242643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 740/1472] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:55:10.547725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c0b3d-271a-4588-a535-5597b8c58aaf None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:10.548209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c0b3d-271a-4588-a535-5597b8c58aaf None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:10.548644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c0b3d-271a-4588-a535-5597b8c58aaf None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:10.551162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 733/1473] 10.222.0.61 () {54 vars in 967 bytes} [Thu Sep 1 13:55:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:55:10.557604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26033e06-6670-4c8e-9f57-bef750c763ee None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:10.557939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26033e06-6670-4c8e-9f57-bef750c763ee None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:10.558215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26033e06-6670-4c8e-9f57-bef750c763ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:10.646352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-26033e06-6670-4c8e-9f57-bef750c763ee None None] MFA Rules not processed for user `42f33454f4b94b7fb54e83252ce564db`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:10.647593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-26033e06-6670-4c8e-9f57-bef750c763ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:10.671530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:10.736693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 741/1474] 10.222.0.61 () {58 vars in 1064 bytes} [Thu Sep 1 13:55:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:55:10.744642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c0b3d-271a-4588-a535-5597b8c58aaf None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:55:10.746235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cd9c0b3d-271a-4588-a535-5597b8c58aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:10.866491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c0b3d-271a-4588-a535-5597b8c58aaf None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:55:10.952516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c30d2-8050-4ab8-a9fa-d37e31fbfa51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:55:10.953133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c30d2-8050-4ab8-a9fa-d37e31fbfa51 service glance] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:10.953410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c30d2-8050-4ab8-a9fa-d37e31fbfa51 service glance] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:10.953678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c30d2-8050-4ab8-a9fa-d37e31fbfa51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:10.955561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-025c30d2-8050-4ab8-a9fa-d37e31fbfa51 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:11.016494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:11.078878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c30d2-8050-4ab8-a9fa-d37e31fbfa51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:55:11.080347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c30d2-8050-4ab8-a9fa-d37e31fbfa51 service glance] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:55:11.102044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 734/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:55:13.243306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ab972-b320-42a6-bb0e-734e99b84c59 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:13.244125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ab972-b320-42a6-bb0e-734e99b84c59 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:13.244125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ab972-b320-42a6-bb0e-734e99b84c59 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:13.245542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 742/1476] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:55:13.251954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc8c5-cee7-4c2b-8c75-77e1aa9d5afa None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:13.252558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc8c5-cee7-4c2b-8c75-77e1aa9d5afa None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:13.252558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc8c5-cee7-4c2b-8c75-77e1aa9d5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:13.347276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-cd6dc8c5-cee7-4c2b-8c75-77e1aa9d5afa None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:13.349054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cd6dc8c5-cee7-4c2b-8c75-77e1aa9d5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:13.479843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 735/1477] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:55:13.764189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ab972-b320-42a6-bb0e-734e99b84c59 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:55:13.765933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-686ab972-b320-42a6-bb0e-734e99b84c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:13.915627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ab972-b320-42a6-bb0e-734e99b84c59 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:55:13.976528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:14.030841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564e69-87de-4051-af04-e255c89ae226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:55:14.031377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564e69-87de-4051-af04-e255c89ae226 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:14.031650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564e69-87de-4051-af04-e255c89ae226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:14.031919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564e69-87de-4051-af04-e255c89ae226 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:14.032582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564e69-87de-4051-af04-e255c89ae226 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:55:14.034437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-3a564e69-87de-4051-af04-e255c89ae226 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:55:14.034437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 13:55:14.034437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 13:55:14.035641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564e69-87de-4051-af04-e255c89ae226 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:55:14.058238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 743/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Sep 1 13:55:13 2022] GET /identity/v3/projects => generated 2553 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 13:55:16.116055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b514eb-98ee-4a46-bd40-3fd13d90557d None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:16.116673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b514eb-98ee-4a46-bd40-3fd13d90557d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:16.116673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b514eb-98ee-4a46-bd40-3fd13d90557d None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:16.118042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 736/1479] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:55:16.124190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7397c-0c73-4207-b2e7-83b39fe95169 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:16.124819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7397c-0c73-4207-b2e7-83b39fe95169 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:16.124819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7397c-0c73-4207-b2e7-83b39fe95169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:16.207180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-82b7397c-0c73-4207-b2e7-83b39fe95169 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:16.208457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-82b7397c-0c73-4207-b2e7-83b39fe95169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:16.364249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 744/1480] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:55:16.672112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b514eb-98ee-4a46-bd40-3fd13d90557d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:55:16.673638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-54b514eb-98ee-4a46-bd40-3fd13d90557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:16.684918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:16.858375 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b514eb-98ee-4a46-bd40-3fd13d90557d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:55:16.948743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:16.988696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968fd44-6551-4643-a43b-0c022e046feb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:55:16.989350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968fd44-6551-4643-a43b-0c022e046feb service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:16.990004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968fd44-6551-4643-a43b-0c022e046feb service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:16.990282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968fd44-6551-4643-a43b-0c022e046feb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:16.992097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f968fd44-6551-4643-a43b-0c022e046feb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:17.098488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968fd44-6551-4643-a43b-0c022e046feb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:55:17.100464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968fd44-6551-4643-a43b-0c022e046feb service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:55:17.122233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 737/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:55:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:55:19.359955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fbd6a-be27-4042-8bad-d49892834126 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:19.359955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fbd6a-be27-4042-8bad-d49892834126 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:19.360448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fbd6a-be27-4042-8bad-d49892834126 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:19.361661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 745/1482] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:55:19.367766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79dd7-e41c-4487-8b44-bc15ef836fa8 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:19.368587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79dd7-e41c-4487-8b44-bc15ef836fa8 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:19.368587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79dd7-e41c-4487-8b44-bc15ef836fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:19.440955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-42f79dd7-e41c-4487-8b44-bc15ef836fa8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:19.443059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-42f79dd7-e41c-4487-8b44-bc15ef836fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:19.496690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:19.561859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 738/1483] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:55:19.854144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fbd6a-be27-4042-8bad-d49892834126 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:55:19.855871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ca2fbd6a-be27-4042-8bad-d49892834126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:19.885154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:19.977693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fbd6a-be27-4042-8bad-d49892834126 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:55:20.072457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56ad7d-91ae-499c-9448-ea519d8265ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:55:20.073071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56ad7d-91ae-499c-9448-ea519d8265ba service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:20.073348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56ad7d-91ae-499c-9448-ea519d8265ba service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:20.073627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56ad7d-91ae-499c-9448-ea519d8265ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:20.075294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ff56ad7d-91ae-499c-9448-ea519d8265ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:20.172418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:20.221667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56ad7d-91ae-499c-9448-ea519d8265ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:55:20.223768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56ad7d-91ae-499c-9448-ea519d8265ba service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:55:20.245198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 746/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:55:22.383204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413b036-e17d-49bd-98b2-1dd892c499eb None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:22.383767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413b036-e17d-49bd-98b2-1dd892c499eb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:22.383767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413b036-e17d-49bd-98b2-1dd892c499eb None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:22.385086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 739/1485] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:55:22.390376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8efab0-fbed-45ac-b442-087c5aedb2d5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:22.390814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8efab0-fbed-45ac-b442-087c5aedb2d5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:22.390916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8efab0-fbed-45ac-b442-087c5aedb2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:22.462497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3a8efab0-fbed-45ac-b442-087c5aedb2d5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:22.463740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3a8efab0-fbed-45ac-b442-087c5aedb2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:22.566651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 747/1486] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:55:22.859553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413b036-e17d-49bd-98b2-1dd892c499eb None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:55:22.861380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a413b036-e17d-49bd-98b2-1dd892c499eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:22.980882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413b036-e17d-49bd-98b2-1dd892c499eb None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:55:23.064776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:23.067310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc8396-27ff-4d68-b429-f0c0cacdf5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:55:23.067933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc8396-27ff-4d68-b429-f0c0cacdf5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:23.068210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc8396-27ff-4d68-b429-f0c0cacdf5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:23.068476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc8396-27ff-4d68-b429-f0c0cacdf5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:23.070244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-15bc8396-27ff-4d68-b429-f0c0cacdf5d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:23.200168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc8396-27ff-4d68-b429-f0c0cacdf5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:55:23.201664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc8396-27ff-4d68-b429-f0c0cacdf5d7 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:55:23.227659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 740/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:55:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:55:25.409571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9511b-d8c1-4304-a3f2-190ab5035d56 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:25.410231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9511b-d8c1-4304-a3f2-190ab5035d56 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:25.410231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9511b-d8c1-4304-a3f2-190ab5035d56 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:25.411587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 748/1488] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:55:25.420116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d28d11-b1d6-4cc2-ade2-b179c6b8b20e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:25.420983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d28d11-b1d6-4cc2-ade2-b179c6b8b20e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:25.420983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d28d11-b1d6-4cc2-ade2-b179c6b8b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:25.496146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-09d28d11-b1d6-4cc2-ade2-b179c6b8b20e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:25.497674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-09d28d11-b1d6-4cc2-ade2-b179c6b8b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:25.559761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:25.612278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 741/1489] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:55:25.910466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9511b-d8c1-4304-a3f2-190ab5035d56 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:55:25.912002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-61d9511b-d8c1-4304-a3f2-190ab5035d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:25.960334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:26.047341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9511b-d8c1-4304-a3f2-190ab5035d56 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:55:26.125123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed6bfe-5ec0-470b-a738-565ff75771ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:55:26.125724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed6bfe-5ec0-470b-a738-565ff75771ce service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:26.126032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed6bfe-5ec0-470b-a738-565ff75771ce service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:26.126301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed6bfe-5ec0-470b-a738-565ff75771ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:26.127913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-57ed6bfe-5ec0-470b-a738-565ff75771ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:26.214189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:26.244963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed6bfe-5ec0-470b-a738-565ff75771ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:55:26.246458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed6bfe-5ec0-470b-a738-565ff75771ce service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:55:26.267987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 749/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:55:28.619421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79645f-3167-4def-a424-d90ba4257b8f None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:28.620006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79645f-3167-4def-a424-d90ba4257b8f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:28.620006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79645f-3167-4def-a424-d90ba4257b8f None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:28.621478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 742/1491] 10.222.0.61 () {54 vars in 927 bytes} [Thu Sep 1 13:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:55:28.628334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427158d-724a-4399-96b3-83b644e47ed6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:28.629386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427158d-724a-4399-96b3-83b644e47ed6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:28.629536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427158d-724a-4399-96b3-83b644e47ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:28.700766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9427158d-724a-4399-96b3-83b644e47ed6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:55:28.702060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9427158d-724a-4399-96b3-83b644e47ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:28.803794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 750/1492] 10.222.0.61 () {58 vars in 1024 bytes} [Thu Sep 1 13:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:55:29.092042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79645f-3167-4def-a424-d90ba4257b8f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:55:29.093620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-be79645f-3167-4def-a424-d90ba4257b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:29.201000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79645f-3167-4def-a424-d90ba4257b8f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:55:29.278134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c606f-88b4-4e99-9c13-18bac8291b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:55:29.278747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c606f-88b4-4e99-9c13-18bac8291b52 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:55:29.279021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c606f-88b4-4e99-9c13-18bac8291b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:55:29.279290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c606f-88b4-4e99-9c13-18bac8291b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:55:29.280930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f16c606f-88b4-4e99-9c13-18bac8291b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:55:29.291892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:29.402060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c606f-88b4-4e99-9c13-18bac8291b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:55:29.403604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c606f-88b4-4e99-9c13-18bac8291b52 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:55:29.423867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 743/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:55:38.801985 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=82 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:55:39.421388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=50 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:55.864281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:55.864938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:55.864938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:55.955529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:55.956838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:56.071932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 751/1494] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 13:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:56:56.082140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b3b5e-ccde-43e3-baf2-a0bf3caf2f71 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:56.082689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b3b5e-ccde-43e3-baf2-a0bf3caf2f71 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:56.082689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b3b5e-ccde-43e3-baf2-a0bf3caf2f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:56.164688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2b5b3b5e-ccde-43e3-baf2-a0bf3caf2f71 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:56.165882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2b5b3b5e-ccde-43e3-baf2-a0bf3caf2f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:56.272250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 744/1495] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 13:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:56.278389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:56.279605 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:56.291329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933bd3f-5850-4408-9afe-dcbe41882fd7 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:56.356020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:56.395082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:56.395760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:56.396037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:56.396308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:56.396963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 13:56:56.397266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:56.398996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:56:56.398996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 13:56:56.398996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 13:56:56.400417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:56.413072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 752/1496] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 13:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:56:56.421712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b3b5e-ccde-43e3-baf2-a0bf3caf2f71 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:56.431622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b3b5e-ccde-43e3-baf2-a0bf3caf2f71 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:56.503977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:56.567542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:56.568529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:56.569002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:56.569487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:56.570651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:56.572915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING py.warnings [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:56:56.572915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: warnings.warn(msg) Sep 01 13:56:56.572915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: Sep 01 13:56:56.574894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:56.755761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 745/1497] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 13:56:56 2022] POST /identity/v3/projects => generated 379 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 13:56:56.760020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:56.761556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:56.802445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78da7a8-65f2-400c-ae88-06ca411b1dc2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:56.905096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:56.905890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:56.906062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:56.906253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:56.907572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:56.908672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:56:56.908672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 13:56:56.908672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 13:56:56.909923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:57.061030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 753/1498] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 13:56:56 2022] POST /identity/v3/users => generated 290 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 13:56:57.062368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:57.063981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:57.131480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51392c54-b9c1-46af-8f70-b4f644c9294b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:57.241032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:57.253166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:57.253810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.254080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.254350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.255009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:57.256020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING py.warnings [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:56:57.256020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: warnings.warn(msg) Sep 01 13:56:57.256020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: Sep 01 13:56:57.256916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:57.270754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 746/1499] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 13:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 13:56:57.277220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:57.279904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:57.291639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b5a78-f844-493d-aa0e-f3d11fbe77f5 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:57.319548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:57.391845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb552371-3a9a-43b6-adbf-f94df3cd107f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:57.392668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552371-3a9a-43b6-adbf-f94df3cd107f admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.393345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552371-3a9a-43b6-adbf-f94df3cd107f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.393711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552371-3a9a-43b6-adbf-f94df3cd107f admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.394456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb552371-3a9a-43b6-adbf-f94df3cd107f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:57.395564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-bb552371-3a9a-43b6-adbf-f94df3cd107f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:56:57.395564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 13:56:57.395564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 13:56:57.396756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb552371-3a9a-43b6-adbf-f94df3cd107f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:57.411407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 754/1500] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 13:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 13:56:57.416291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:57.418415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:57.455484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e052d02-b928-4d5a-87c0-745bf19a5adc admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:57.558581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c62128-5d5e-47b5-a448-4c544e9c8c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:57.559260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62128-5d5e-47b5-a448-4c544e9c8c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.559541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62128-5d5e-47b5-a448-4c544e9c8c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.559818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62128-5d5e-47b5-a448-4c544e9c8c72 admin admin] PATH_INFO: `/v3/projects/2d7495def3524087a290ca1ea11774f6/users/95492fe9f0854dd59bb451dee186e1e3/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.585536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c62128-5d5e-47b5-a448-4c544e9c8c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7495def3524087a290ca1ea11774f6, user_id=95492fe9f0854dd59bb451dee186e1e3, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:57.586639 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING py.warnings [None req-e2c62128-5d5e-47b5-a448-4c544e9c8c72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:56:57.586639 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: warnings.warn(msg) Sep 01 13:56:57.586639 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: Sep 01 13:56:57.588673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c62128-5d5e-47b5-a448-4c544e9c8c72 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:57.679308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 747/1501] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 13:56:57 2022] PUT /identity/v3/projects/2d7495def3524087a290ca1ea11774f6/users/95492fe9f0854dd59bb451dee186e1e3/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:56:57.693832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a92e2-63ea-44a6-a9d5-d45785cea0d0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.694158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a92e2-63ea-44a6-a9d5-d45785cea0d0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.694452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a92e2-63ea-44a6-a9d5-d45785cea0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.759927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-843a92e2-63ea-44a6-a9d5-d45785cea0d0 None None] MFA Rules not processed for user `95492fe9f0854dd59bb451dee186e1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:57.761188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-843a92e2-63ea-44a6-a9d5-d45785cea0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:57.851695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 755/1502] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:57.865715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5384a68-2b37-42bc-8dc8-a221f4fac12b None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.866028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5384a68-2b37-42bc-8dc8-a221f4fac12b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.866304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5384a68-2b37-42bc-8dc8-a221f4fac12b None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.867838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 748/1503] 10.222.0.61 () {54 vars in 964 bytes} [Thu Sep 1 13:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:57.873806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439957a1-7521-416e-adae-70fb6c94adc0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:57.874109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439957a1-7521-416e-adae-70fb6c94adc0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:57.874385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439957a1-7521-416e-adae-70fb6c94adc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:57.909244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:57.962626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-439957a1-7521-416e-adae-70fb6c94adc0 None None] MFA Rules not processed for user `caa058ab479540848d2d9793fdfe4e09`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:57.963852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-439957a1-7521-416e-adae-70fb6c94adc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:58.056074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 756/1504] 10.222.0.61 () {58 vars in 1061 bytes} [Thu Sep 1 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:58.066622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5384a68-2b37-42bc-8dc8-a221f4fac12b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:58.068730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f5384a68-2b37-42bc-8dc8-a221f4fac12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:58.129298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:58.192834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5384a68-2b37-42bc-8dc8-a221f4fac12b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:58.286509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:58.287152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:58.287432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:58.287705 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:58.289362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:58.406573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:58.408559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:58.412324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:58.429680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 749/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:56:58.476040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06c5b-8747-4ca1-a516-e15f4a6e6dc1 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:58.476322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06c5b-8747-4ca1-a516-e15f4a6e6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:58.476595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06c5b-8747-4ca1-a516-e15f4a6e6dc1 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:58.477912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 757/1506] 10.222.0.61 () {52 vars in 830 bytes} [Thu Sep 1 13:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:56:58.481665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:58.482909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:58.584982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9868cf-9fb9-418f-84c6-4d1b848e0633 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:58.682651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:58.683184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:58.683462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:58.683725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] PATH_INFO: `/v3/users/95492fe9f0854dd59bb451dee186e1e3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:58.696316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95492fe9f0854dd59bb451dee186e1e3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:58.697396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING py.warnings [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:56:58.697396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: warnings.warn(msg) Sep 01 13:56:58.697396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: Sep 01 13:56:58.698606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:58.709260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:58.902250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95492fe9f0854dd59bb451dee186e1e3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:56:58.916854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95492fe9f0854dd59bb451dee186e1e3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:56:58.917577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95492fe9f0854dd59bb451dee186e1e3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:56:58.918037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95492fe9f0854dd59bb451dee186e1e3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:56:58.985485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95492fe9f0854dd59bb451dee186e1e3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:56:59.017998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 750/1507] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 13:56:58 2022] DELETE /identity/v3/users/95492fe9f0854dd59bb451dee186e1e3 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:56:59.023317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:59.023834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:59.023941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:59.084629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:56:59.085890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:59.190002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:59.213523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 758/1508] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 13:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:56:59.223440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:56:59.224865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:59.364484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee1676-bb9a-4bfd-b38a-975c17944ec2 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:56:59.428574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:56:59.454518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4d41d-4ebb-4d92-b067-6b281607ab00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:56:59.455615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4d41d-4ebb-4d92-b067-6b281607ab00 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:56:59.456157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4d41d-4ebb-4d92-b067-6b281607ab00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:56:59.456662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4d41d-4ebb-4d92-b067-6b281607ab00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:56:59.459944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c2e4d41d-4ebb-4d92-b067-6b281607ab00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:56:59.589999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4d41d-4ebb-4d92-b067-6b281607ab00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:56:59.591476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4d41d-4ebb-4d92-b067-6b281607ab00 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:56:59.616575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 751/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:57:00.158136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:57:00.159417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:57:00.171892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ada930-d171-4da6-8c1d-f1b1955cd060 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:57:00.286250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:57:00.287181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:57:00.287657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:57:00.287968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] PATH_INFO: `/v3/projects/2d7495def3524087a290ca1ea11774f6` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:57:00.307388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d7495def3524087a290ca1ea11774f6)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:57:00.309306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 13:57:00.309306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 13:57:00.309306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 13:57:00.311823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:57:00.453273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d7495def3524087a290ca1ea11774f6', 'request_id': 'req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:57:00.460492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d7495def3524087a290ca1ea11774f6', 'request_id': 'req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:57:00.560360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d7495def3524087a290ca1ea11774f6', 'request_id': 'req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:57:00.596695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] The token cache is being invalidate because project 2d7495def3524087a290ca1ea11774f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 13:57:00.596984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:57:00.597436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b09a0f0b-3ce4-47b0-b1bd-9923b06db899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 13:57:00.599176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 759/1510] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 13:57:00 2022] DELETE /identity/v3/projects/2d7495def3524087a290ca1ea11774f6 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 13:57:09.614131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=62 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:57:10.593717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:31.303189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc247d-8ed0-487a-8243-4cccb1b8c8fc None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:31.304165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc247d-8ed0-487a-8243-4cccb1b8c8fc None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:31.304165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc247d-8ed0-487a-8243-4cccb1b8c8fc None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:31.305189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:31.312459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6856afc-3f45-4740-9738-9b15a52b7b6b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:31.313045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6856afc-3f45-4740-9738-9b15a52b7b6b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:31.313416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6856afc-3f45-4740-9738-9b15a52b7b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:31.463294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e6856afc-3f45-4740-9738-9b15a52b7b6b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:31.465411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e6856afc-3f45-4740-9738-9b15a52b7b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:31.654105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:31.814553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc247d-8ed0-487a-8243-4cccb1b8c8fc None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:31.818080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cadc247d-8ed0-487a-8243-4cccb1b8c8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:31.947980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc247d-8ed0-487a-8243-4cccb1b8c8fc None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:31.954188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88354e03-e67a-4b00-b78c-0bf9debad8fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:31.954803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354e03-e67a-4b00-b78c-0bf9debad8fe service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:31.955084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354e03-e67a-4b00-b78c-0bf9debad8fe service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:31.955361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354e03-e67a-4b00-b78c-0bf9debad8fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:31.964105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88354e03-e67a-4b00-b78c-0bf9debad8fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:31.965724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88354e03-e67a-4b00-b78c-0bf9debad8fe service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:31.976444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 753/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:32.676846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc6f98-f8de-42b4-b84e-804ca543142e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:32.677656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc6f98-f8de-42b4-b84e-804ca543142e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:32.677656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc6f98-f8de-42b4-b84e-804ca543142e None None] PATH_INFO: `/v3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:32.679298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 761/1514] 10.222.0.46 () {54 vars in 925 bytes} [Thu Sep 1 13:58:32 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 13:58:32.687769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e352e8-7772-4657-8f4f-fa2ec5171626 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:32.688434 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e352e8-7772-4657-8f4f-fa2ec5171626 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:32.688904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e352e8-7772-4657-8f4f-fa2ec5171626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:32.777296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-83e352e8-7772-4657-8f4f-fa2ec5171626 None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:32.778957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-83e352e8-7772-4657-8f4f-fa2ec5171626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:32.862180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:32.868121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 754/1515] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 13:58:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:32.889005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c0d89-f7cd-4a4b-929f-66aa0ee346bb None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:32.889290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c0d89-f7cd-4a4b-929f-66aa0ee346bb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:32.889567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c0d89-f7cd-4a4b-929f-66aa0ee346bb None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:32.890976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 762/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:32.896589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2de1b-3f34-4fff-aeb7-197212048ba4 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:32.896871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2de1b-3f34-4fff-aeb7-197212048ba4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:32.897151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2de1b-3f34-4fff-aeb7-197212048ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:32.918457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-62a2de1b-3f34-4fff-aeb7-197212048ba4 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:32.919700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-62a2de1b-3f34-4fff-aeb7-197212048ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:32.928727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 755/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:33.084427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c0d89-f7cd-4a4b-929f-66aa0ee346bb None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:33.100266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c0d89-f7cd-4a4b-929f-66aa0ee346bb None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:33.106467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df2056-46e8-4531-a7c0-1d2b74aa8e49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:33.107215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df2056-46e8-4531-a7c0-1d2b74aa8e49 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:33.108153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df2056-46e8-4531-a7c0-1d2b74aa8e49 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:33.108538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df2056-46e8-4531-a7c0-1d2b74aa8e49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:33.118053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df2056-46e8-4531-a7c0-1d2b74aa8e49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:33.119721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df2056-46e8-4531-a7c0-1d2b74aa8e49 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:33.130931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 763/1518] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:42.154914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f0f46-3834-481d-bbbf-25305886f226 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:42.155909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f0f46-3834-481d-bbbf-25305886f226 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:42.155909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f0f46-3834-481d-bbbf-25305886f226 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:42.157192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 756/1519] 10.222.0.61 () {54 vars in 961 bytes} [Thu Sep 1 13:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:42.163027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac9991-3ede-4487-a13a-20b1fe3bbd95 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:42.163587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac9991-3ede-4487-a13a-20b1fe3bbd95 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:42.163587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac9991-3ede-4487-a13a-20b1fe3bbd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:42.184663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-15ac9991-3ede-4487-a13a-20b1fe3bbd95 None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:42.185984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-15ac9991-3ede-4487-a13a-20b1fe3bbd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:42.194942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 764/1520] 10.222.0.61 () {58 vars in 1058 bytes} [Thu Sep 1 13:58:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:42.203345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f0f46-3834-481d-bbbf-25305886f226 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:42.218319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f0f46-3834-481d-bbbf-25305886f226 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:42.223528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4db33-0c5e-4524-9632-869871aefd27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:42.224150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4db33-0c5e-4524-9632-869871aefd27 service placement] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:42.224429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4db33-0c5e-4524-9632-869871aefd27 service placement] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:42.224703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4db33-0c5e-4524-9632-869871aefd27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:42.234216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4db33-0c5e-4524-9632-869871aefd27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:42.235835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4db33-0c5e-4524-9632-869871aefd27 service placement] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:42.247551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 757/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:44.383927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9a460-24f9-46f3-825e-f054cf67ac2f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:44.384515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9a460-24f9-46f3-825e-f054cf67ac2f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:44.384653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9a460-24f9-46f3-825e-f054cf67ac2f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:44.386103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:44.392488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae91bc1-8f7e-4758-b439-e720ce7884cd None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:44.393106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae91bc1-8f7e-4758-b439-e720ce7884cd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:44.393106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae91bc1-8f7e-4758-b439-e720ce7884cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:44.414009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-aae91bc1-8f7e-4758-b439-e720ce7884cd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:44.415236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-aae91bc1-8f7e-4758-b439-e720ce7884cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:44.423515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:44.583384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9a460-24f9-46f3-825e-f054cf67ac2f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:44.601394 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9a460-24f9-46f3-825e-f054cf67ac2f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:44.609265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cd60b-dfb6-4dea-b2a1-5fe3ef4e39a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:44.610380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd60b-dfb6-4dea-b2a1-5fe3ef4e39a4 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:44.610928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd60b-dfb6-4dea-b2a1-5fe3ef4e39a4 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:44.611438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cd60b-dfb6-4dea-b2a1-5fe3ef4e39a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:44.629041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cd60b-dfb6-4dea-b2a1-5fe3ef4e39a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:44.632231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cd60b-dfb6-4dea-b2a1-5fe3ef4e39a4 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:44.651317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 766/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:45.499577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd15c0-7b21-44cf-9505-3fe23fb6e934 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:45.500126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd15c0-7b21-44cf-9505-3fe23fb6e934 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:45.500246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd15c0-7b21-44cf-9505-3fe23fb6e934 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:45.501612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 13:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:45.507026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3e3d0-709d-4b88-8ec0-93b058997ee9 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:45.507602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3e3d0-709d-4b88-8ec0-93b058997ee9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:45.507602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3e3d0-709d-4b88-8ec0-93b058997ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:45.528332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-5fd3e3d0-709d-4b88-8ec0-93b058997ee9 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:45.530215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5fd3e3d0-709d-4b88-8ec0-93b058997ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:45.538425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 13:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:45.692624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd15c0-7b21-44cf-9505-3fe23fb6e934 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:45.707770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd15c0-7b21-44cf-9505-3fe23fb6e934 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:45.713242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ac600-27d1-4741-a61b-e87c2d746670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:45.713879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac600-27d1-4741-a61b-e87c2d746670 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:45.714159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac600-27d1-4741-a61b-e87c2d746670 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:45.714423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac600-27d1-4741-a61b-e87c2d746670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:45.723531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ac600-27d1-4741-a61b-e87c2d746670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:45.725592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ac600-27d1-4741-a61b-e87c2d746670 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:45.736273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 760/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:49.966839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228a043-8113-4747-b79b-bc7f34f753a8 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:49.967462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228a043-8113-4747-b79b-bc7f34f753a8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:49.967462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228a043-8113-4747-b79b-bc7f34f753a8 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:49.968754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:49.975721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bf7ee-f3db-46e9-9783-b256e6c27d8d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:49.976378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bf7ee-f3db-46e9-9783-b256e6c27d8d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:49.976517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bf7ee-f3db-46e9-9783-b256e6c27d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:50.000657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c66bf7ee-f3db-46e9-9783-b256e6c27d8d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:50.002411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c66bf7ee-f3db-46e9-9783-b256e6c27d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:50.013455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:50.303909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228a043-8113-4747-b79b-bc7f34f753a8 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:50.305707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c228a043-8113-4747-b79b-bc7f34f753a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:50.341175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:58:50.391753 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228a043-8113-4747-b79b-bc7f34f753a8 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:50.396047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87245115-34f1-4673-981d-524ef571c7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:50.396669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87245115-34f1-4673-981d-524ef571c7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:50.396946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87245115-34f1-4673-981d-524ef571c7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:50.397214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87245115-34f1-4673-981d-524ef571c7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:50.407178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87245115-34f1-4673-981d-524ef571c7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:50.408610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87245115-34f1-4673-981d-524ef571c7a4 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:50.418702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 769/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:58:52.360350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf731e8-f6ae-4f01-8ad7-3b598598aace None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:52.360962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf731e8-f6ae-4f01-8ad7-3b598598aace None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:52.360962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf731e8-f6ae-4f01-8ad7-3b598598aace None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:52.362239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 13:58:52.368221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283e8a-25f4-4b56-9b5c-cdf7b15c2de6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:52.368777 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283e8a-25f4-4b56-9b5c-cdf7b15c2de6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:52.368777 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283e8a-25f4-4b56-9b5c-cdf7b15c2de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:52.391055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a6283e8a-25f4-4b56-9b5c-cdf7b15c2de6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:52.392291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a6283e8a-25f4-4b56-9b5c-cdf7b15c2de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:52.400609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:52.690546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf731e8-f6ae-4f01-8ad7-3b598598aace None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:52.692768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0cf731e8-f6ae-4f01-8ad7-3b598598aace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:52.708327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf731e8-f6ae-4f01-8ad7-3b598598aace None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:52.711996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896400b-c8b6-45d8-9d2b-66a405d944dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:52.712622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896400b-c8b6-45d8-9d2b-66a405d944dc service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:52.712896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896400b-c8b6-45d8-9d2b-66a405d944dc service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:52.713168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896400b-c8b6-45d8-9d2b-66a405d944dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:52.723909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896400b-c8b6-45d8-9d2b-66a405d944dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:52.725367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896400b-c8b6-45d8-9d2b-66a405d944dc service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:52.735979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 763/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 13:58:56.413831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1451c-b9e9-44af-ab92-6b4f211ba295 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:56.414384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1451c-b9e9-44af-ab92-6b4f211ba295 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:56.414384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1451c-b9e9-44af-ab92-6b4f211ba295 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:56.415661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:56.422188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb13be-a1ca-4c38-ad89-3f29e77c23fe None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:56.422767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb13be-a1ca-4c38-ad89-3f29e77c23fe None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:56.422767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb13be-a1ca-4c38-ad89-3f29e77c23fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:56.448483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-abdb13be-a1ca-4c38-ad89-3f29e77c23fe None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:56.449733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-abdb13be-a1ca-4c38-ad89-3f29e77c23fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:56.458557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 13:58:57.024757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1451c-b9e9-44af-ab92-6b4f211ba295 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:57.026445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0ae1451c-b9e9-44af-ab92-6b4f211ba295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:57.106226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1451c-b9e9-44af-ab92-6b4f211ba295 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:57.110460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730498c5-b0c9-44da-92ff-b828fd7c3136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:57.116583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730498c5-b0c9-44da-92ff-b828fd7c3136 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:57.116583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730498c5-b0c9-44da-92ff-b828fd7c3136 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:57.116583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730498c5-b0c9-44da-92ff-b828fd7c3136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:57.120618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730498c5-b0c9-44da-92ff-b828fd7c3136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:57.122379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730498c5-b0c9-44da-92ff-b828fd7c3136 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:57.131795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 772/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:58:59.084992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dca7dc-6f12-491c-af90-ae68f82a0a7d None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:59.085572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dca7dc-6f12-491c-af90-ae68f82a0a7d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:59.085759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dca7dc-6f12-491c-af90-ae68f82a0a7d None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:59.087457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 13:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 13:58:59.095813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d6cd-10a2-4e94-b1a1-c93d4b613269 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:59.096404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d6cd-10a2-4e94-b1a1-c93d4b613269 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:59.096569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d6cd-10a2-4e94-b1a1-c93d4b613269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:59.119151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-1896d6cd-10a2-4e94-b1a1-c93d4b613269 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 13:58:59.120573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1896d6cd-10a2-4e94-b1a1-c93d4b613269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:59.129442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 13:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 13:58:59.695925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dca7dc-6f12-491c-af90-ae68f82a0a7d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 13:58:59.697645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d7dca7dc-6f12-491c-af90-ae68f82a0a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 13:58:59.712118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dca7dc-6f12-491c-af90-ae68f82a0a7d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 13:58:59.716785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d35f9-7049-4081-b140-3340527c83b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 13:58:59.717712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d35f9-7049-4081-b140-3340527c83b7 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 13:58:59.718591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d35f9-7049-4081-b140-3340527c83b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 13:58:59.719201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d35f9-7049-4081-b140-3340527c83b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 13:58:59.729491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d35f9-7049-4081-b140-3340527c83b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 13:58:59.730986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d35f9-7049-4081-b140-3340527c83b7 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 13:58:59.740868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 766/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 13:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 13:59:09.113165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=63 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 13:59:09.736562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=37 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:25.420235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec428b0-16c3-4d54-be10-5a9f309962f6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.420901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec428b0-16c3-4d54-be10-5a9f309962f6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.421046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec428b0-16c3-4d54-be10-5a9f309962f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.452867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccc636-251e-4105-8dd9-8e360d651755 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.454568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccc636-251e-4105-8dd9-8e360d651755 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.455228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccc636-251e-4105-8dd9-8e360d651755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.462227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-eec428b0-16c3-4d54-be10-5a9f309962f6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.464540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-eec428b0-16c3-4d54-be10-5a9f309962f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:25.490886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3bccc636-251e-4105-8dd9-8e360d651755 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.492942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3bccc636-251e-4105-8dd9-8e360d651755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:25.493255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 774/1540] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:25.513694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 767/1541] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:25.516233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a09af8-5f28-4c96-b8c3-84075b95faf7 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.516580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a09af8-5f28-4c96-b8c3-84075b95faf7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.516876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a09af8-5f28-4c96-b8c3-84075b95faf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.523212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5460c-f175-406c-a64f-497973c38858 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.523826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5460c-f175-406c-a64f-497973c38858 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.524344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5460c-f175-406c-a64f-497973c38858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.553419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-94a09af8-5f28-4c96-b8c3-84075b95faf7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.553419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-94a09af8-5f28-4c96-b8c3-84075b95faf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:25.557458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c9c5460c-f175-406c-a64f-497973c38858 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.559664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c9c5460c-f175-406c-a64f-497973c38858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:25.565176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 775/1542] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:25.571518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a09af8-5f28-4c96-b8c3-84075b95faf7 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:25.577731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 768/1543] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:25.582438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5460c-f175-406c-a64f-497973c38858 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:25.588606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a09af8-5f28-4c96-b8c3-84075b95faf7 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:25.593740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5460c-f175-406c-a64f-497973c38858 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:25.600305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377ce01-79eb-4d95-9602-a58c26666b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:25.601349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ce01-79eb-4d95-9602-a58c26666b00 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.601992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ce01-79eb-4d95-9602-a58c26666b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.602482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ce01-79eb-4d95-9602-a58c26666b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.603698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377ce01-79eb-4d95-9602-a58c26666b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:25.604243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377ce01-79eb-4d95-9602-a58c26666b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:25.606930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377ce01-79eb-4d95-9602-a58c26666b00 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:25.608713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:25.609604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.610140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.610794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.611783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:25.612341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:25.613608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING py.warnings [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:00:25.613608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: warnings.warn(msg) Sep 01 14:00:25.613608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: Sep 01 14:00:25.615312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:25.635065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 769/1544] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:00:25.641250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 776/1545] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:00:25.644565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:25.647241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bb235-eca9-4454-b2dc-036149e9deda None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.647241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bb235-eca9-4454-b2dc-036149e9deda None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.647241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bb235-eca9-4454-b2dc-036149e9deda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.659398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5b237-8894-4026-b2fe-d789208b39f6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:25.673741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a75837-bbe3-4f68-b298-6b7cbead4217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:25.675367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a75837-bbe3-4f68-b298-6b7cbead4217 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.675893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a75837-bbe3-4f68-b298-6b7cbead4217 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.676414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a75837-bbe3-4f68-b298-6b7cbead4217 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.677337 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a75837-bbe3-4f68-b298-6b7cbead4217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:25.679600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a75837-bbe3-4f68-b298-6b7cbead4217 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:25.683730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7b5bb235-eca9-4454-b2dc-036149e9deda None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.685750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7b5bb235-eca9-4454-b2dc-036149e9deda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:25.704436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 777/1546] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:25.705317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bb235-eca9-4454-b2dc-036149e9deda None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:25.721572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bb235-eca9-4454-b2dc-036149e9deda None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:25.733425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cf0c7-feb4-46c2-81c4-8f8cb0fa2bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:25.734103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cf0c7-feb4-46c2-81c4-8f8cb0fa2bdc admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.734463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cf0c7-feb4-46c2-81c4-8f8cb0fa2bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.734736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cf0c7-feb4-46c2-81c4-8f8cb0fa2bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.735413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cf0c7-feb4-46c2-81c4-8f8cb0fa2bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:25.736508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-a33cf0c7-feb4-46c2-81c4-8f8cb0fa2bdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:00:25.736508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 14:00:25.736508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 14:00:25.737746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cf0c7-feb4-46c2-81c4-8f8cb0fa2bdc admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:25.800625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 770/1547] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:00:25.803098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33688d-8289-4abd-9cc4-0482fd213a0f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.803594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33688d-8289-4abd-9cc4-0482fd213a0f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.804080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33688d-8289-4abd-9cc4-0482fd213a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.847217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-df33688d-8289-4abd-9cc4-0482fd213a0f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.860302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-df33688d-8289-4abd-9cc4-0482fd213a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:25.892374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 778/1548] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:25.894805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f4b04-000f-45b7-b247-c44709a79356 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.895490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f4b04-000f-45b7-b247-c44709a79356 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.896011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f4b04-000f-45b7-b247-c44709a79356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.923690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-fe4f4b04-000f-45b7-b247-c44709a79356 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.927214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fe4f4b04-000f-45b7-b247-c44709a79356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:25.970826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 771/1549] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:25.972189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf998d8-1be2-4cf8-8171-23516a79247d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:25.972479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf998d8-1be2-4cf8-8171-23516a79247d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:25.972786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf998d8-1be2-4cf8-8171-23516a79247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:25.997200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-0cf998d8-1be2-4cf8-8171-23516a79247d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:25.998469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0cf998d8-1be2-4cf8-8171-23516a79247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.031590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 779/1550] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.032759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f4b04-000f-45b7-b247-c44709a79356 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.046256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f4b04-000f-45b7-b247-c44709a79356 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.053320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff5f45-4e3c-4371-a259-e0ccef4f3b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.054322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff5f45-4e3c-4371-a259-e0ccef4f3b12 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.054749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff5f45-4e3c-4371-a259-e0ccef4f3b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.055132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff5f45-4e3c-4371-a259-e0ccef4f3b12 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.056193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff5f45-4e3c-4371-a259-e0ccef4f3b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.059581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff5f45-4e3c-4371-a259-e0ccef4f3b12 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.080520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 772/1551] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.081971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b9d6-d4f6-47f4-a88f-3b8ceebdc6e5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.082319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b9d6-d4f6-47f4-a88f-3b8ceebdc6e5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.082658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35b9d6-d4f6-47f4-a88f-3b8ceebdc6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.105077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-be35b9d6-d4f6-47f4-a88f-3b8ceebdc6e5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.105077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-be35b9d6-d4f6-47f4-a88f-3b8ceebdc6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.113803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 773/1552] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.117634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35b9d6-d4f6-47f4-a88f-3b8ceebdc6e5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.128245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35b9d6-d4f6-47f4-a88f-3b8ceebdc6e5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.133078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9612311-356d-4f74-b889-364dc3d6cda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.133686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612311-356d-4f74-b889-364dc3d6cda0 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.133988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612311-356d-4f74-b889-364dc3d6cda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.134255 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612311-356d-4f74-b889-364dc3d6cda0 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.134891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9612311-356d-4f74-b889-364dc3d6cda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.135902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING py.warnings [None req-b9612311-356d-4f74-b889-364dc3d6cda0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:00:26.135902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: warnings.warn(msg) Sep 01 14:00:26.135902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: Sep 01 14:00:26.137192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9612311-356d-4f74-b889-364dc3d6cda0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.223238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 780/1553] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:26.224550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bffb2-d72a-4bc7-a147-e5a24fc89a8e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.224839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bffb2-d72a-4bc7-a147-e5a24fc89a8e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.225111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bffb2-d72a-4bc7-a147-e5a24fc89a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.246740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 774/1554] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:26.246965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3e0bffb2-d72a-4bc7-a147-e5a24fc89a8e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.251816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642f47c-c49e-4d7a-bbb5-ed501b33d837 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.251950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3e0bffb2-d72a-4bc7-a147-e5a24fc89a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.252036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642f47c-c49e-4d7a-bbb5-ed501b33d837 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.252036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642f47c-c49e-4d7a-bbb5-ed501b33d837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.259120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 781/1555] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.259872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bffb2-d72a-4bc7-a147-e5a24fc89a8e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.270366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bffb2-d72a-4bc7-a147-e5a24fc89a8e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.276836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c642f47c-c49e-4d7a-bbb5-ed501b33d837 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.279351 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c642f47c-c49e-4d7a-bbb5-ed501b33d837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.280935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418b524-cc96-4173-bfe6-7577164c2777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.281815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418b524-cc96-4173-bfe6-7577164c2777 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.282097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418b524-cc96-4173-bfe6-7577164c2777 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.282369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418b524-cc96-4173-bfe6-7577164c2777 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.283027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418b524-cc96-4173-bfe6-7577164c2777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:26.283324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418b524-cc96-4173-bfe6-7577164c2777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.284955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418b524-cc96-4173-bfe6-7577164c2777 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.288539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 775/1556] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.290936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06de122-4a61-4bef-8619-9981457bc0ac None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.291610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06de122-4a61-4bef-8619-9981457bc0ac None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.292243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06de122-4a61-4bef-8619-9981457bc0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.298969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 782/1557] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:26.300392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67ee13-2c99-4283-85a3-619310e5bdf9 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.309139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67ee13-2c99-4283-85a3-619310e5bdf9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.309139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67ee13-2c99-4283-85a3-619310e5bdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.318828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c06de122-4a61-4bef-8619-9981457bc0ac None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.320068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c06de122-4a61-4bef-8619-9981457bc0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.328526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0e67ee13-2c99-4283-85a3-619310e5bdf9 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.328729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 776/1558] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.329744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0e67ee13-2c99-4283-85a3-619310e5bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.330013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d0d8c-2a30-4eac-aaf0-e18f1fa8accd None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.330303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d0d8c-2a30-4eac-aaf0-e18f1fa8accd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.330583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d0d8c-2a30-4eac-aaf0-e18f1fa8accd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.339287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 783/1559] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.339287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67ee13-2c99-4283-85a3-619310e5bdf9 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.347346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:26.350668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67ee13-2c99-4283-85a3-619310e5bdf9 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.353130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-242d0d8c-2a30-4eac-aaf0-e18f1fa8accd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.354388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-242d0d8c-2a30-4eac-aaf0-e18f1fa8accd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.355625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7f4c4-b717-46c8-b1c9-796ed05f8d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.356766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f4c4-b717-46c8-b1c9-796ed05f8d83 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.357046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f4c4-b717-46c8-b1c9-796ed05f8d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.357375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f4c4-b717-46c8-b1c9-796ed05f8d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.358068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7f4c4-b717-46c8-b1c9-796ed05f8d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.359693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7f4c4-b717-46c8-b1c9-796ed05f8d83 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.366092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 777/1560] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.368280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d0d8c-2a30-4eac-aaf0-e18f1fa8accd None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.371585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 784/1561] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:26.374846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0d764-bb0a-4f03-bbe7-ee15a53fe19c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.374846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0d764-bb0a-4f03-bbe7-ee15a53fe19c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.375128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0d764-bb0a-4f03-bbe7-ee15a53fe19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.380625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d0d8c-2a30-4eac-aaf0-e18f1fa8accd None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.385532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.386196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.386469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.386737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.387913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.389585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.396434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-36a0d764-bb0a-4f03-bbe7-ee15a53fe19c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.397699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-36a0d764-bb0a-4f03-bbe7-ee15a53fe19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.403698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 778/1562] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:26.403698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.407571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 785/1563] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.414618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0d764-bb0a-4f03-bbe7-ee15a53fe19c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.414792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2af965-5678-4e48-befb-7442505d7ae4 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.419853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0d764-bb0a-4f03-bbe7-ee15a53fe19c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.420326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1192-a488-4899-9768-6cf0f6ec636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.420949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1192-a488-4899-9768-6cf0f6ec636d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.421222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1192-a488-4899-9768-6cf0f6ec636d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.421488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1192-a488-4899-9768-6cf0f6ec636d admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.422177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e1192-a488-4899-9768-6cf0f6ec636d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:26.422474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e1192-a488-4899-9768-6cf0f6ec636d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.424085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e1192-a488-4899-9768-6cf0f6ec636d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.424945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.425565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.425944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.426330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.426979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.428905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.436998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 779/1564] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:26.439287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd5ef5-d687-46b0-be1d-fa3ee534dee5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.439287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd5ef5-d687-46b0-be1d-fa3ee534dee5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.439287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd5ef5-d687-46b0-be1d-fa3ee534dee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.458558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4cbd5ef5-d687-46b0-be1d-fa3ee534dee5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:26.459749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4cbd5ef5-d687-46b0-be1d-fa3ee534dee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:26.468317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 780/1565] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:26.469101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd5ef5-d687-46b0-be1d-fa3ee534dee5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.478391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd5ef5-d687-46b0-be1d-fa3ee534dee5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.482914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.483488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.483765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.484054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.484687 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:26.484981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.486589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.498516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 781/1566] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:26.499315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.509865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819954a2-50dd-4b36-8fdc-6b3fa43b127d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.516713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.516713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.516713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.516713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.517205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:26.517412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.519099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.532854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 782/1567] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:26.533931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.545882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 786/1568] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:26.546989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.549517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a40536-b2b1-47da-bf45-515372b9f6a8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.576636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:26.609695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.610263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.610542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.610821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.611464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.613074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.627242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 783/1569] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:26.627242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.627654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95ac35-0b71-4a83-a640-c9774a66cb1a admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.630317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.630933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.631207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.631475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.632128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.633736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.635092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646dafa-1bfe-42d2-9d99-7c53e8daf81e admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.640135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.641008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.641008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.641276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.641959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:26.642260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.643853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.649560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 787/1570] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:26.650457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.657459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 784/1571] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:26.658329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.665729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b745664-3a74-4c55-95bb-31e67c3238e6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.668013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fef02-0feb-4cb9-9cb9-45e539f2e52c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.670579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.671204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.671480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.671747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.672377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.672591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.673172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.673448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.673724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.674319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.674444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:26.674683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.676267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.687759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 785/1572] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:26.688544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.698295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b931bab-77da-427b-b7c1-406d0c141d65 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.703379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.703973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.711014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.711014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.711014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.711014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.801669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 788/1573] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:26.802509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.819343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 786/1574] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:26.819343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:26.872472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65763cd1-9564-4fb9-8cd8-eeb69ef70378 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.873753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:26.885184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88b2e6-dd8d-4ddf-9a24-2eb656273374 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:26.891630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.892254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.892545 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.892819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.893457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.895596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:26.932341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:26.932963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:26.933243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:26.933514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:26.934711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:26.936652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.022756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 787/1575] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.023803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.034592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c378-43b3-4bee-a07f-10c5c91ddf44 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.041688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.042408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.042692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.042969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] PATH_INFO: `/v3/projects/8352a8d484e24264a536b8960dab338f/users/1943057b69d342d9b45fc02be9f3d5aa/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.057755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 789/1576] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:26 2022] POST /identity/v3/projects => generated 403 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.057755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.064019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8352a8d484e24264a536b8960dab338f, user_id=1943057b69d342d9b45fc02be9f3d5aa, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.066849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.094482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:27.130037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1b8f1-2cda-407f-97a9-ed7f05b2dd23 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.134736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.135414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.135690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.135959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] PATH_INFO: `/v3/projects/1c53449be8e448b09a184ea79278bb2a/users/a70246b8e8b842129c99827859b90478/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.151617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c53449be8e448b09a184ea79278bb2a, user_id=a70246b8e8b842129c99827859b90478, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.159518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:00:27.159518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 14:00:27.159518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 14:00:27.159518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.160122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 788/1577] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:27 2022] PUT /identity/v3/projects/8352a8d484e24264a536b8960dab338f/users/1943057b69d342d9b45fc02be9f3d5aa/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:27.160122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.230324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c1cd5-d565-47e9-acaa-fe946660b82b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.235666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.236277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.236560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.236832 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.237470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.239510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.248153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 790/1578] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:27 2022] PUT /identity/v3/projects/1c53449be8e448b09a184ea79278bb2a/users/a70246b8e8b842129c99827859b90478/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:27.253954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.319484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86adf90-b3d1-4dcf-95b6-42d7071d2731 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.324590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.325204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.325487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.325762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.326432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.328597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.345956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 789/1579] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/projects => generated 389 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.347445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.420556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb3730-d42f-4602-bf3e-533ef43a6d0c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.425400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.426135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.426406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.426780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.427429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.429389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.453748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 791/1580] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.453748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.524854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908cefb-b1d7-4ded-8b1d-34ae77e93f8c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.531309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefbecf-5dfa-4996-ad42-7d056acc6855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.532111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefbecf-5dfa-4996-ad42-7d056acc6855 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.532504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefbecf-5dfa-4996-ad42-7d056acc6855 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.532885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefbecf-5dfa-4996-ad42-7d056acc6855 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.533689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefbecf-5dfa-4996-ad42-7d056acc6855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.538928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefbecf-5dfa-4996-ad42-7d056acc6855 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.539137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 790/1581] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.539137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.550482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac373236-9f5b-4d20-aa9e-8bf760180a15 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.555949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.556689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.557014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.557335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.558196 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.562552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.575177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 791/1582] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:27.575177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.583752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcd76d-1af5-4854-a998-637905b1a043 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.589844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.591540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.592014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.592573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.593867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.597683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.619176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 792/1583] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.621972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adef7a-892f-4fa2-81d5-b5f479f28433 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.622450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adef7a-892f-4fa2-81d5-b5f479f28433 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.622921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adef7a-892f-4fa2-81d5-b5f479f28433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.645832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-13adef7a-892f-4fa2-81d5-b5f479f28433 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:27.648601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-13adef7a-892f-4fa2-81d5-b5f479f28433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:27.656287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 793/1584] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:27.657689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa32a-bce1-4229-b10e-8184ca08d93e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.658037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa32a-bce1-4229-b10e-8184ca08d93e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.658313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa32a-bce1-4229-b10e-8184ca08d93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.680576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-df3fa32a-bce1-4229-b10e-8184ca08d93e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:27.681864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-df3fa32a-bce1-4229-b10e-8184ca08d93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:27.691046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 794/1585] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:27.692191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa32a-bce1-4229-b10e-8184ca08d93e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.704130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa32a-bce1-4229-b10e-8184ca08d93e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.710526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.711331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.711759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.712223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.713051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.716138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.735725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 792/1586] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/users => generated 302 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.735725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.735725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:27.736178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:27.736900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687f6dd-ccf9-4dba-bc96-0bd49cf0aaa9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.745317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.746701 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.747198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.747519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.748524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.751528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.833364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 795/1587] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.834258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.835746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 793/1588] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:27 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:27.837283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.848960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4c30f-e858-4c65-8f87-c6eb2ca792be admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.850261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed260f-9d8b-475d-8b24-6d1980821b46 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.854565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.857509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.857755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.857755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.858577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.858695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.858793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.858793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.858961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.859098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.859234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.859641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.870649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 794/1589] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:27.876390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 796/1590] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:27.876669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.876815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.885660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2cd3-afca-4bd5-a19f-1428c1a28b79 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.891018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.891631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.891911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.892188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.892839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.894543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.905949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 795/1591] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:27.906782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.955571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a97bd-3f9b-485d-97d1-c14ac7bb6ded admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.960114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:27.960854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:27.961299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:27.961710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:27.974501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:27.975996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:27.988884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 797/1592] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:27.989806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:27.990290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc49518-23cc-43d7-bb2d-1f84dec493cb admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:27.997124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.002818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.002818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.002818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.003572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc677d7c-1291-4e10-83ed-7b986b3fca12 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.007407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.008133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.008496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.008828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.009953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.011543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.023266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.025009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.032006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 798/1593] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.032006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.039818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 796/1594] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:28.040142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.046713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25db4b-dd05-4dd6-abec-a02374317458 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.051563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddb9cc-30a3-4067-b2b6-5db6e41c4b72 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.056403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.057642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.058024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.058335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.058887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.059257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.059737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.060159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.060425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.061374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.063231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.063975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.079861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 797/1595] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.080658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 799/1596] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.081269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.081797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.093030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e7db1-9656-44be-b123-bf5d03e74cb7 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.094550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53085b-a80c-4125-9920-677e1bff2345 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.100544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.100968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.111417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.111417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.111835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.112005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] PATH_INFO: `/v3/projects/69c0c03f491744169f699453bbb5ca86/users/caba7ce953dc442d9f1b6cac52d4ef55/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.112169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.112169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.112169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.112169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.120296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c0c03f491744169f699453bbb5ca86, user_id=caba7ce953dc442d9f1b6cac52d4ef55, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.123998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.129824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 800/1597] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.129824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.138040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdccd0-d9fd-42ce-8cce-cfde6e9307f3 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.142739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.143379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.143638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.143910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.144560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.146198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.157053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 801/1598] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.161638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.169175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc819bf-71bb-4aec-9b45-3bbb56bae893 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.178313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.178313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.178313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.178313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.179281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.179281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.189755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 802/1599] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.191004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.201035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae011065-d49d-41ac-b97f-ef8205a98e12 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.205669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.206957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.206957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.206957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.207440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.209033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.220559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 803/1600] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.221889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.233079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86279e01-1ed8-4764-a2a8-37094369d992 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.238377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.239084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.239365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.239638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.240298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.242206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.254125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 804/1601] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:28.255121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.267006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ba084-86f0-47cf-bb0c-f531f289ff79 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.291354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 798/1602] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:28 2022] PUT /identity/v3/projects/69c0c03f491744169f699453bbb5ca86/users/caba7ce953dc442d9f1b6cac52d4ef55/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:28.292391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.360601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.361340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.361625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.361934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] PATH_INFO: `/v3/projects/1e427cb0d55f41de8e5c3a019be848bf/users/7a3807832b0b413d866d8a3b01b2b5d0/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.381017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e427cb0d55f41de8e5c3a019be848bf, user_id=7a3807832b0b413d866d8a3b01b2b5d0, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.383963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.386579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:28.388715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245b400-8c11-48e8-b405-d55cafb37325 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.393792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.394556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.394836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.395111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] PATH_INFO: `/v3/projects/421b15cacc7a46388f87a068db634585/users/219806fd01d14fa58f336ea720f274b1/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.415502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421b15cacc7a46388f87a068db634585, user_id=219806fd01d14fa58f336ea720f274b1, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.418931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.541264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 805/1603] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:28 2022] PUT /identity/v3/projects/1e427cb0d55f41de8e5c3a019be848bf/users/7a3807832b0b413d866d8a3b01b2b5d0/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:28.542251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.542800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 799/1604] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:28 2022] PUT /identity/v3/projects/421b15cacc7a46388f87a068db634585/users/219806fd01d14fa58f336ea720f274b1/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:28.545563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.580982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:28.642714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b6e31-2c26-4a7d-b491-e635a5e43122 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.643145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0111d-e864-4265-9504-9f30fd3a1735 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.647775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14026fed-0dd2-4c15-a3f6-40ca45c0ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.648658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026fed-0dd2-4c15-a3f6-40ca45c0ad76 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.648992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026fed-0dd2-4c15-a3f6-40ca45c0ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.649205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026fed-0dd2-4c15-a3f6-40ca45c0ad76 admin admin] PATH_INFO: `/v3/projects/048ede8fff72416da06a3a99f9ed9370/users/1bf9aa4f1cc0435e9983244256a2374b/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.649791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:28.650694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.650975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.651250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] PATH_INFO: `/v3/projects/c19855b809cc43b2925dc3001284baaa/users/c0afa15991f14e0ca0d3701d06a89049/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.668051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14026fed-0dd2-4c15-a3f6-40ca45c0ad76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=048ede8fff72416da06a3a99f9ed9370, user_id=1bf9aa4f1cc0435e9983244256a2374b, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.670956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14026fed-0dd2-4c15-a3f6-40ca45c0ad76 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.679426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19855b809cc43b2925dc3001284baaa, user_id=c0afa15991f14e0ca0d3701d06a89049, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:28.682282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:28.831348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 800/1605] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:28 2022] PUT /identity/v3/projects/c19855b809cc43b2925dc3001284baaa/users/c0afa15991f14e0ca0d3701d06a89049/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:28.833078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 806/1606] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:28 2022] PUT /identity/v3/projects/048ede8fff72416da06a3a99f9ed9370/users/1bf9aa4f1cc0435e9983244256a2374b/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:28.833213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:28.833993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac1b2b-025b-4c3f-a39c-5c2dd1e1faec None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.834438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac1b2b-025b-4c3f-a39c-5c2dd1e1faec None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.835137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac1b2b-025b-4c3f-a39c-5c2dd1e1faec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.877323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-acac1b2b-025b-4c3f-a39c-5c2dd1e1faec None None] MFA Rules not processed for user `caba7ce953dc442d9f1b6cac52d4ef55`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:28.879442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-acac1b2b-025b-4c3f-a39c-5c2dd1e1faec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:28.930879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42507139-faef-4aa7-b277-a3959881b2d4 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:28.966834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 807/1607] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:28.969475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8360-dc2f-44dc-a75a-19e68569c605 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:28.970145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8360-dc2f-44dc-a75a-19e68569c605 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:28.976337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8360-dc2f-44dc-a75a-19e68569c605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:28.999898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb5cb8-b1f1-4e42-8d59-6566e8d0e404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:29.001506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5cb8-b1f1-4e42-8d59-6566e8d0e404 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.001814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5cb8-b1f1-4e42-8d59-6566e8d0e404 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.002205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5cb8-b1f1-4e42-8d59-6566e8d0e404 admin admin] PATH_INFO: `/v3/projects/7e322f2f7da74039aff258b70af12095/users/b77f38582b8842d7a303e8431299191a/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.018260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-fedf8360-dc2f-44dc-a75a-19e68569c605 None None] MFA Rules not processed for user `219806fd01d14fa58f336ea720f274b1`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:29.019570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fedf8360-dc2f-44dc-a75a-19e68569c605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:29.023048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb5cb8-b1f1-4e42-8d59-6566e8d0e404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e322f2f7da74039aff258b70af12095, user_id=b77f38582b8842d7a303e8431299191a, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:29.026158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb5cb8-b1f1-4e42-8d59-6566e8d0e404 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:29.094010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:29.106093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 808/1608] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:29.107624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f926a-97e9-4da5-b87f-5a984367866c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.107911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f926a-97e9-4da5-b87f-5a984367866c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.108183 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f926a-97e9-4da5-b87f-5a984367866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.147195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 801/1609] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:28 2022] PUT /identity/v3/projects/7e322f2f7da74039aff258b70af12095/users/b77f38582b8842d7a303e8431299191a/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:29.148779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-846f926a-97e9-4da5-b87f-5a984367866c None None] MFA Rules not processed for user `7a3807832b0b413d866d8a3b01b2b5d0`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:29.150156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-846f926a-97e9-4da5-b87f-5a984367866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:29.150946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f50a2-2a2a-4e6a-9ef4-ddead406b4fa None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.151492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f50a2-2a2a-4e6a-9ef4-ddead406b4fa None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.152388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f50a2-2a2a-4e6a-9ef4-ddead406b4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.207241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3b9f50a2-2a2a-4e6a-9ef4-ddead406b4fa None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:29.208675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3b9f50a2-2a2a-4e6a-9ef4-ddead406b4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:29.265741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:29.276878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 809/1610] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:29.282726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78991d-2f0c-42e0-90f4-25ea43b538c7 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.294009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78991d-2f0c-42e0-90f4-25ea43b538c7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.294009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78991d-2f0c-42e0-90f4-25ea43b538c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.322388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4c78991d-2f0c-42e0-90f4-25ea43b538c7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:29.325899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4c78991d-2f0c-42e0-90f4-25ea43b538c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:29.347590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 802/1611] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:29.349738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cde94-3717-4d64-8ec0-b3f06b1010a6 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.350057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cde94-3717-4d64-8ec0-b3f06b1010a6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.350335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cde94-3717-4d64-8ec0-b3f06b1010a6 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.356972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 803/1612] 10.222.0.61 () {54 vars in 962 bytes} [Thu Sep 1 14:00:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:29.358752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6759-f2fd-47ba-903b-f31eb2a2059b None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.371725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6759-f2fd-47ba-903b-f31eb2a2059b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.371725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6759-f2fd-47ba-903b-f31eb2a2059b None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.371725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 804/1613] 10.222.0.61 () {54 vars in 962 bytes} [Thu Sep 1 14:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:29.371725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa38e6-61d8-498b-9881-9dd7c24b1701 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.371725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa38e6-61d8-498b-9881-9dd7c24b1701 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.371725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa38e6-61d8-498b-9881-9dd7c24b1701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.397437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b6fa38e6-61d8-498b-9881-9dd7c24b1701 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:29.399675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b6fa38e6-61d8-498b-9881-9dd7c24b1701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:29.417854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 805/1614] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:29.417854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa38e6-61d8-498b-9881-9dd7c24b1701 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:29.432247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa38e6-61d8-498b-9881-9dd7c24b1701 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:29.440459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce44cb-bb26-416c-b06e-fe73f0c306f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:29.441492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce44cb-bb26-416c-b06e-fe73f0c306f4 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.442040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce44cb-bb26-416c-b06e-fe73f0c306f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.442549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce44cb-bb26-416c-b06e-fe73f0c306f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.442776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 810/1615] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:29.444246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daded5-1976-4026-af2b-57698601ace8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.445958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce44cb-bb26-416c-b06e-fe73f0c306f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:29.450132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce44cb-bb26-416c-b06e-fe73f0c306f4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:29.458221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daded5-1976-4026-af2b-57698601ace8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.462114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daded5-1976-4026-af2b-57698601ace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.538453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b3daded5-1976-4026-af2b-57698601ace8 None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:29.539761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b3daded5-1976-4026-af2b-57698601ace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:29.614397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 806/1616] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/projects => generated 403 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:29.618006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f264a9-b78c-4724-9216-f85b896dbafa None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.618527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f264a9-b78c-4724-9216-f85b896dbafa None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.619081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f264a9-b78c-4724-9216-f85b896dbafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.657355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 811/1617] 10.222.0.61 () {58 vars in 1059 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:29.659576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daded5-1976-4026-af2b-57698601ace8 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:29.663186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c9f264a9-b78c-4724-9216-f85b896dbafa None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:29.664702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c9f264a9-b78c-4724-9216-f85b896dbafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:29.728914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daded5-1976-4026-af2b-57698601ace8 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:29.758964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 807/1618] 10.222.0.61 () {58 vars in 1059 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:29.760157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f264a9-b78c-4724-9216-f85b896dbafa None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:29.790311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:29.790928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.791210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.791483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.826055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f264a9-b78c-4724-9216-f85b896dbafa None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:29.830204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347ca63-3573-4243-beac-726096630473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:29.830823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347ca63-3573-4243-beac-726096630473 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.831109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347ca63-3573-4243-beac-726096630473 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.831369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347ca63-3573-4243-beac-726096630473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.883018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:29.884538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:29.897498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 812/1619] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:29.898538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:29.919928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ba0b2-0fe4-4f76-ba5f-2c5282a9b428 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:29.922550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:29.923305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.923690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.924101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.929975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347ca63-3573-4243-beac-726096630473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:29.931683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347ca63-3573-4243-beac-726096630473 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:29.941411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:29.944066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:29.950334 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 808/1620] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:29.952475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347ca63-3573-4243-beac-726096630473 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:29.958373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 813/1621] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:29.959362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:29.967738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347ca63-3573-4243-beac-726096630473 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:29.972812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95cd3d-530d-4cb6-9cc8-b89386175415 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:29.973976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:29.975571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.975835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.976111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:29.976758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:29.979035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:29.979959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:29.980744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:29.981230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:29.981619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.065282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:30.067107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:30.077519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 814/1622] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:30.078483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:30.092159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9a8ec-9b85-4ab4-80b9-95acfc2704ef service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:30.095378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 809/1623] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:29 2022] POST /identity/v3/users => generated 308 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:30.099460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:30.100096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.100377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:30.100559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.100723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.114249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366f129-0958-46ac-a39e-d034d7232d75 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:30.123955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:30.129286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.129286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.129286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.129286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:30.132638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:30.151445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:30.157551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 810/1624] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:00:30.164338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:30.219916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cf985-54d6-4468-ab1c-51bfe27e3abd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:30.221120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:30.224537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:30.243692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80724d-dbf7-480b-a329-315128141cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:30.245483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 815/1625] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:30.248501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80724d-dbf7-480b-a329-315128141cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.249190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80724d-dbf7-480b-a329-315128141cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.249807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80724d-dbf7-480b-a329-315128141cc0 admin admin] PATH_INFO: `/v3/projects/cf8ddb392aba4047b76fee757945d581/users/da91fa12ae6c49a1a13cabdab458ce62/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.312692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80724d-dbf7-480b-a329-315128141cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8ddb392aba4047b76fee757945d581, user_id=da91fa12ae6c49a1a13cabdab458ce62, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:30.319649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80724d-dbf7-480b-a329-315128141cc0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:30.422562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 811/1626] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:30 2022] PUT /identity/v3/projects/cf8ddb392aba4047b76fee757945d581/users/da91fa12ae6c49a1a13cabdab458ce62/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:00:30.423335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:30.437437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:30.441914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6cc7-882a-437b-ad81-a1ef40e72060 service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:30.569306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b5309-fb26-40e4-9d07-b65b4ee5ca98 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.569981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b5309-fb26-40e4-9d07-b65b4ee5ca98 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.570548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b5309-fb26-40e4-9d07-b65b4ee5ca98 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.574432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 812/1627] 10.222.0.61 () {54 vars in 912 bytes} [Thu Sep 1 14:00:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:30.590664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a9cfb-9937-4bf5-bb68-a6d7a0a8e91a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.592356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a9cfb-9937-4bf5-bb68-a6d7a0a8e91a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.593310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a9cfb-9937-4bf5-bb68-a6d7a0a8e91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.625876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:30.626585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.626862 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.627135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.627812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:30.631398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:30.634083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-036a9cfb-9937-4bf5-bb68-a6d7a0a8e91a None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:30.636462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-036a9cfb-9937-4bf5-bb68-a6d7a0a8e91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:30.657125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 816/1628] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:30.659929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:30.676323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a8ea9-2ae5-40fb-a796-c5e3593093c1 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:30.681714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cbb54-34ec-42c6-8785-0836c5e7396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:30.686965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cbb54-34ec-42c6-8785-0836c5e7396b admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.686965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cbb54-34ec-42c6-8785-0836c5e7396b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.686965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cbb54-34ec-42c6-8785-0836c5e7396b admin admin] PATH_INFO: `/v3/projects/cf8ddb392aba4047b76fee757945d581/users/da91fa12ae6c49a1a13cabdab458ce62/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.686965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cbb54-34ec-42c6-8785-0836c5e7396b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8ddb392aba4047b76fee757945d581, user_id=da91fa12ae6c49a1a13cabdab458ce62, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:30.689802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cbb54-34ec-42c6-8785-0836c5e7396b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:30.713954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 817/1629] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:30 2022] PUT /identity/v3/projects/cf8ddb392aba4047b76fee757945d581/users/da91fa12ae6c49a1a13cabdab458ce62/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:30.726792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d753310-bb8d-4f6c-a159-da56a0eb3787 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.727239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d753310-bb8d-4f6c-a159-da56a0eb3787 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.727622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d753310-bb8d-4f6c-a159-da56a0eb3787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.764761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 813/1630] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:30.766534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1772c4-c50b-4cd4-af50-30211e723ed7 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.766816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1772c4-c50b-4cd4-af50-30211e723ed7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.767089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1772c4-c50b-4cd4-af50-30211e723ed7 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.768753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 814/1631] 10.222.0.61 () {54 vars in 912 bytes} [Thu Sep 1 14:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:30.778091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373346a2-4787-4d0a-b0c2-c001fae67eb0 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:30.778464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373346a2-4787-4d0a-b0c2-c001fae67eb0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:30.778801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373346a2-4787-4d0a-b0c2-c001fae67eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:30.807797 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-1d753310-bb8d-4f6c-a159-da56a0eb3787 None None] MFA Rules not processed for user `da91fa12ae6c49a1a13cabdab458ce62`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:30.809084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1d753310-bb8d-4f6c-a159-da56a0eb3787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:30.810989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-373346a2-4787-4d0a-b0c2-c001fae67eb0 None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:30.812562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-373346a2-4787-4d0a-b0c2-c001fae67eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:30.832599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:30.938693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 818/1632] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:30.939727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d753310-bb8d-4f6c-a159-da56a0eb3787 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:30.944137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 815/1633] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:30.952038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373346a2-4787-4d0a-b0c2-c001fae67eb0 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:31.032320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373346a2-4787-4d0a-b0c2-c001fae67eb0 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:31.037443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:31.038435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:31.038713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:31.038982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:31.053485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:31.055516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:31.071538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 816/1634] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:31.080791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d753310-bb8d-4f6c-a159-da56a0eb3787 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:31.085541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:31.086216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:31.086491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:31.086762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:31.102968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:31.104422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:31.117883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 819/1635] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:31.217609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:31.232312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2428f-4e39-4277-bc00-1f8db84bd90b service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:31.232663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:31.239800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:31.241317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:31.244598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:31.245034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:31.276781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:31.278798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:31.310152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 817/1636] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:31.435456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:31.809830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:31.818595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:31.824105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d4128-8f3c-4d9b-83ab-5d0833da3514 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:31.824105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:31.824105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:31.824105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:31.844372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] PATH_INFO: `/v3/users/7a3807832b0b413d866d8a3b01b2b5d0` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:31.844372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3807832b0b413d866d8a3b01b2b5d0)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:31.844372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: WARNING py.warnings [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:00:31.844372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: warnings.warn(msg) Sep 01 14:00:31.844372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: Sep 01 14:00:31.844372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:31.848053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7724f-1a2d-44bf-b916-ba363471cd9c service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:31.848053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:31.848053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:31.848053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:31.856606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] PATH_INFO: `/v3/users/219806fd01d14fa58f336ea720f274b1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:31.856606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] RBAC: Authorizing `identity:delete_user(user_id=219806fd01d14fa58f336ea720f274b1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:31.856606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:32.210986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219806fd01d14fa58f336ea720f274b1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.212483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3807832b0b413d866d8a3b01b2b5d0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.227447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219806fd01d14fa58f336ea720f274b1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.229355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219806fd01d14fa58f336ea720f274b1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.230288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219806fd01d14fa58f336ea720f274b1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.244614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3807832b0b413d866d8a3b01b2b5d0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.332536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-34b8225c-c987-4f3b-932b-069887b7108d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219806fd01d14fa58f336ea720f274b1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.375350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 818/1637] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:31 2022] DELETE /identity/v3/users/219806fd01d14fa58f336ea720f274b1 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:00:32.381197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:32.382462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:32.382932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:32.420123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:32.421855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:32.474160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3807832b0b413d866d8a3b01b2b5d0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.474855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3807832b0b413d866d8a3b01b2b5d0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.513633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-26a0dbc2-000a-4389-b943-ff884cfb512f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3807832b0b413d866d8a3b01b2b5d0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:32.531293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 820/1638] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:31 2022] DELETE /identity/v3/users/7a3807832b0b413d866d8a3b01b2b5d0 => generated 0 bytes in 904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:32.535942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68251b4c-96d0-44cf-a008-947ec222a84c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:32.536603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68251b4c-96d0-44cf-a008-947ec222a84c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:32.537642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68251b4c-96d0-44cf-a008-947ec222a84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:32.546585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 819/1639] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:32.558659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:32.560422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:32.582134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-68251b4c-96d0-44cf-a008-947ec222a84c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:32.583514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-68251b4c-96d0-44cf-a008-947ec222a84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:32.628757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:32.705464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af368f-a151-4b6b-b39a-ee3d53affa63 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:32.758203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 821/1640] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:32.789111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:32.797845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015c998-8df7-4737-8437-73ea4562bfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:32.799426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c998-8df7-4737-8437-73ea4562bfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:32.800054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c998-8df7-4737-8437-73ea4562bfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:32.803094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c998-8df7-4737-8437-73ea4562bfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:32.828730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015c998-8df7-4737-8437-73ea4562bfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:32.836925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015c998-8df7-4737-8437-73ea4562bfd0 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:32.863838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 820/1641] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:32.916139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68251b4c-96d0-44cf-a008-947ec222a84c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:32.957935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68251b4c-96d0-44cf-a008-947ec222a84c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:33.001119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:33.007761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:33.024296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:33.024800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:33.048172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:33.058019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:33.086976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 822/1642] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:00:34.440407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:34.440881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:34.440950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:34.557340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] MFA Rules not processed for user `1bf9aa4f1cc0435e9983244256a2374b`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:34.557340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:34.604095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:34.606906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:34.632558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e72928-d7c0-401e-8747-3133fb891413 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:34.646089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:34.646841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:34.647130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:34.647406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] PATH_INFO: `/v3/projects/421b15cacc7a46388f87a068db634585` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:34.654785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=421b15cacc7a46388f87a068db634585)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:34.666770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:34.742715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 821/1643] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:34.743824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:34.751234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:34.774822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103b6e-ed3f-465b-9025-8631da4b102c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:34.783291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:34.785199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:34.785710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:34.786222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] PATH_INFO: `/v3/projects/1e427cb0d55f41de8e5c3a019be848bf` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:34.788585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e427cb0d55f41de8e5c3a019be848bf)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:34.791483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: WARNING py.warnings [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 14:00:34.791483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: warnings.warn(msg) Sep 01 14:00:34.791483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: Sep 01 14:00:34.793754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:34.798962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:34.900116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421b15cacc7a46388f87a068db634585', 'request_id': 'req-7b5894d5-222b-4b41-8e80-627ba7762501'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:34.928759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421b15cacc7a46388f87a068db634585', 'request_id': 'req-7b5894d5-222b-4b41-8e80-627ba7762501'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.085162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421b15cacc7a46388f87a068db634585', 'request_id': 'req-7b5894d5-222b-4b41-8e80-627ba7762501'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.113155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] The token cache is being invalidate because project 421b15cacc7a46388f87a068db634585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:35.113155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.113741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.115755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 823/1644] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:34 2022] DELETE /identity/v3/projects/421b15cacc7a46388f87a068db634585 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:00:35.116323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:35.120150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:35.125566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e427cb0d55f41de8e5c3a019be848bf', 'request_id': 'req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.146921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e427cb0d55f41de8e5c3a019be848bf', 'request_id': 'req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.158420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e427cb0d55f41de8e5c3a019be848bf', 'request_id': 'req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.282616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5894d5-222b-4b41-8e80-627ba7762501 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:35.300718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] The token cache is being invalidate because project 1e427cb0d55f41de8e5c3a019be848bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:35.301170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.301959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c2dbcb8e-ae4f-4eab-b0a9-dbb52cab3abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:35.303974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 822/1645] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:34 2022] DELETE /identity/v3/projects/1e427cb0d55f41de8e5c3a019be848bf => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:35.309488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:35.309762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:35.310131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:35.372022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:35.375689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:35.385683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:35.429341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:35.429341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:35.429341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:35.429341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:35.429613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:35.431054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:35.562647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 824/1646] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:35 2022] POST /identity/v3/projects => generated 405 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:35.566344 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:35.571299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:35.666420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 823/1647] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:35.674221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:35.678105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:35.711851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde9ce3-48da-405d-bb4b-e6b9441328e7 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:35.728689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:35.729681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:35.730141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:35.730619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] PATH_INFO: `/v3/users/da91fa12ae6c49a1a13cabdab458ce62` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:35.735091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da91fa12ae6c49a1a13cabdab458ce62)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:35.737370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:35.795955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63c61a-5d3a-47ea-9268-cc6160cd7a43 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:35.816323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e4b87-12e0-4a93-a561-5c7dcfe7118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:35.818358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4b87-12e0-4a93-a561-5c7dcfe7118f admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:35.818675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4b87-12e0-4a93-a561-5c7dcfe7118f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:35.820933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4b87-12e0-4a93-a561-5c7dcfe7118f admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:35.823770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e4b87-12e0-4a93-a561-5c7dcfe7118f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:35.831341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e4b87-12e0-4a93-a561-5c7dcfe7118f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.026012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 824/1648] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:35 2022] POST /identity/v3/users => generated 309 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:36.028404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e4fb-4fdc-4e71-bec0-3639f273cb0a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.028871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e4fb-4fdc-4e71-bec0-3639f273cb0a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.029283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e4fb-4fdc-4e71-bec0-3639f273cb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.047350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da91fa12ae6c49a1a13cabdab458ce62'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:36.057035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da91fa12ae6c49a1a13cabdab458ce62'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:36.100448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c146e4fb-4fdc-4e71-bec0-3639f273cb0a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:36.102668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c146e4fb-4fdc-4e71-bec0-3639f273cb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:36.119429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 825/1649] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:36.120551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e4fb-4fdc-4e71-bec0-3639f273cb0a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:36.153719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e4fb-4fdc-4e71-bec0-3639f273cb0a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:36.154523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da91fa12ae6c49a1a13cabdab458ce62'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:36.155283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da91fa12ae6c49a1a13cabdab458ce62'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:36.167546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:36.170047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.172504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.173055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.175424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:36.179377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da91fa12ae6c49a1a13cabdab458ce62'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:36.185790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.189811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 825/1650] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:35 2022] DELETE /identity/v3/users/da91fa12ae6c49a1a13cabdab458ce62 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:36.194101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:36.198413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:36.222002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 826/1651] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:36.225462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:36.230447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:36.278564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e610186-af5b-4f79-9698-5c36bea42063 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:36.332226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:36.372710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8bb6b-c445-4851-b285-87d28bd7a614 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:36.383986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:36.385002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.385497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.385936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.387322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:36.387770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:36.389547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.405760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:36.418857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:36.418857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.418857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.418857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] PATH_INFO: `/v3/projects/81d15464f866479486c3b593e9440a82/users/41c61d68b9654b23a080d390f7768df9/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.419623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 826/1652] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:36.425505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:36.443987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a990a-a100-485d-ac2e-418353868a29 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:36.447712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d15464f866479486c3b593e9440a82, user_id=41c61d68b9654b23a080d390f7768df9, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:36.453316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.454190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:36.455007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.455413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.455813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.456610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:36.458785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.560903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 827/1653] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:36 2022] PUT /identity/v3/projects/81d15464f866479486c3b593e9440a82/users/41c61d68b9654b23a080d390f7768df9/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:36.565620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:36.591727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 827/1654] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:36 2022] POST /identity/v3/projects => generated 391 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:36.593581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:36.675346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ccc88-d348-451a-bd6f-b0badc5f1239 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:36.750767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffeb67-2a6c-4606-b5d1-be799a8e2467 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:36.769582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:36.770478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.770888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.771306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.772092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:36.774238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.775565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:36.776439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.776820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.777230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.778282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:36.781872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.797823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 828/1655] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:36.809390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:36.845970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f02aa-50e9-4451-9169-606370228902 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:36.856712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:36.859184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:36.859753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:36.860170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] PATH_INFO: `/v3/projects/81d15464f866479486c3b593e9440a82/users/41c61d68b9654b23a080d390f7768df9/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:36.863853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d15464f866479486c3b593e9440a82, user_id=41c61d68b9654b23a080d390f7768df9, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:36.868014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:36.885890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 829/1656] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:36 2022] PUT /identity/v3/projects/81d15464f866479486c3b593e9440a82/users/41c61d68b9654b23a080d390f7768df9/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:36.936982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 828/1657] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:36 2022] POST /identity/v3/users => generated 296 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:36.948881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:37.107078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:37.108872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:37.124500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b215c-9559-45e2-ae54-d91b52cf872e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:37.224878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15e06c-e2ac-4b14-a09a-c4364f241f62 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:37.234656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:37.236503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:37.236798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:37.237081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:37.237807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:37.239899 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:37.261089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 830/1658] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:37.265463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:37.266128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:37.267172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:37.267478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:37.267881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] PATH_INFO: `/v3/users/caba7ce953dc442d9f1b6cac52d4ef55` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:37.269481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=caba7ce953dc442d9f1b6cac52d4ef55)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:37.287490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db826c-0bdb-4d8f-be0f-a5cea1fb077c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:37.294039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:37.300101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:37.301389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:37.301389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:37.301620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:37.302330 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:37.303967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:37.331536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 831/1659] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:00:37.335833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:37.355765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752aba3-cb9e-470c-81f9-8432c4297200 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:37.364658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f71ab1-ba2f-4eb2-912d-755ac0ba7668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:37.365105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f71ab1-ba2f-4eb2-912d-755ac0ba7668 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:37.365386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f71ab1-ba2f-4eb2-912d-755ac0ba7668 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:37.365664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f71ab1-ba2f-4eb2-912d-755ac0ba7668 admin admin] PATH_INFO: `/v3/projects/49f8266d74614e14ae10be9b90535ff9/users/5afb1b8bdba54493bd9e56d899798c56/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:37.401466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f71ab1-ba2f-4eb2-912d-755ac0ba7668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f8266d74614e14ae10be9b90535ff9, user_id=5afb1b8bdba54493bd9e56d899798c56, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:37.404328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f71ab1-ba2f-4eb2-912d-755ac0ba7668 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:37.519818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 832/1660] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:37 2022] PUT /identity/v3/projects/49f8266d74614e14ae10be9b90535ff9/users/5afb1b8bdba54493bd9e56d899798c56/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:00:37.533721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aa19b-04ac-418c-b904-92d16ba3c1ea None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:37.534078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aa19b-04ac-418c-b904-92d16ba3c1ea None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:37.534355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aa19b-04ac-418c-b904-92d16ba3c1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:37.633002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f45aa19b-04ac-418c-b904-92d16ba3c1ea None None] MFA Rules not processed for user `5afb1b8bdba54493bd9e56d899798c56`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:37.634492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f45aa19b-04ac-418c-b904-92d16ba3c1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:37.687427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caba7ce953dc442d9f1b6cac52d4ef55'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:37.704098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caba7ce953dc442d9f1b6cac52d4ef55'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:37.776740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caba7ce953dc442d9f1b6cac52d4ef55'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:37.778002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caba7ce953dc442d9f1b6cac52d4ef55'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:37.795464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 833/1661] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:37.797271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45aa19b-04ac-418c-b904-92d16ba3c1ea None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:37.802878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ec889d99-53ce-4471-9c48-c6f12c907d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caba7ce953dc442d9f1b6cac52d4ef55'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:37.823443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 829/1662] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:37 2022] DELETE /identity/v3/users/caba7ce953dc442d9f1b6cac52d4ef55 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:00:37.828654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:37.829174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:37.829669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:37.871969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:37.871969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:38.037696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45aa19b-04ac-418c-b904-92d16ba3c1ea None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:38.116163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 830/1663] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:38.133020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:38.135305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:38.153954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653402e-c154-4ff4-ac9f-79fd76557f34 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:38.160242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:38.161162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:38.161685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:38.162085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] PATH_INFO: `/v3/projects/cf8ddb392aba4047b76fee757945d581` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:38.164684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8ddb392aba4047b76fee757945d581)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:38.167164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:38.209695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:38.210278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:38.210552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:38.210832 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:38.211503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:38.213728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:38.329638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:38.346859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8ddb392aba4047b76fee757945d581', 'request_id': 'req-818f7739-e3b8-4796-a31a-049161fecbb7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:38.361371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8ddb392aba4047b76fee757945d581', 'request_id': 'req-818f7739-e3b8-4796-a31a-049161fecbb7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:38.402244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 834/1664] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:37 2022] POST /identity/v3/projects => generated 393 bytes in 605 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:00:38.404906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:38.407023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:38.463135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8ddb392aba4047b76fee757945d581', 'request_id': 'req-818f7739-e3b8-4796-a31a-049161fecbb7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:38.484489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:38.507533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] The token cache is being invalidate because project cf8ddb392aba4047b76fee757945d581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:38.509092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:38.514212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:38.520632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 831/1665] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:38 2022] DELETE /identity/v3/projects/cf8ddb392aba4047b76fee757945d581 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:38.529485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:38.533397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:38.615645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea2821-6595-45d1-bc72-cd57c421a12f admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:38.627677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b0636-9ed3-4646-aff9-14e36104f977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:38.629544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b0636-9ed3-4646-aff9-14e36104f977 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:38.630549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b0636-9ed3-4646-aff9-14e36104f977 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:38.630839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b0636-9ed3-4646-aff9-14e36104f977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:38.637370 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e90b0636-9ed3-4646-aff9-14e36104f977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:38.732480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f7739-e3b8-4796-a31a-049161fecbb7 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:38.850436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b0636-9ed3-4646-aff9-14e36104f977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:38.859734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b0636-9ed3-4646-aff9-14e36104f977 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:38.880250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 835/1666] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:38.881630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdc6c9-d0fd-443c-84ba-e5397572b735 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:38.882233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdc6c9-d0fd-443c-84ba-e5397572b735 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:38.882233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdc6c9-d0fd-443c-84ba-e5397572b735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:38.894332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:38.895004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:38.895239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:38.895548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:38.896154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:38.898785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:38.922519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-32cdc6c9-d0fd-443c-84ba-e5397572b735 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:38.924652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-32cdc6c9-d0fd-443c-84ba-e5397572b735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:38.936300 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 836/1667] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:38.948750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eb81e-2b9a-4eb8-a54c-880aa6ab362d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:38.949391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eb81e-2b9a-4eb8-a54c-880aa6ab362d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:38.949915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eb81e-2b9a-4eb8-a54c-880aa6ab362d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:38.981836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-854eb81e-2b9a-4eb8-a54c-880aa6ab362d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:38.991883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-854eb81e-2b9a-4eb8-a54c-880aa6ab362d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:39.004146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 837/1668] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:39.012547 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854eb81e-2b9a-4eb8-a54c-880aa6ab362d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.022943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 832/1669] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:38 2022] POST /identity/v3/users => generated 303 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:39.028330 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854eb81e-2b9a-4eb8-a54c-880aa6ab362d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.028503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.038599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.039618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.040116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.040604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.041474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:39.042053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.044522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.051012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48f6ae-0af4-4ec0-886d-1c673e81fadd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.063956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 838/1670] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:39.064431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.065154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.065434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.065706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.066404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.068464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.072860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.089412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14529a9a-d050-4863-abe5-6d949e074761 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.097149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.099199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.099716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.100191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.101152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.101884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 833/1671] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:39.102889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.104132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.129373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8101a-a264-4fa8-912e-c1b65a4adb12 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.146681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.147390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.147672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.147947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] PATH_INFO: `/v3/projects/626826d727a94b1ca378c3bc99f8ab15/users/4a0090947e4f447887c306cc2a565c1c/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.182318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626826d727a94b1ca378c3bc99f8ab15, user_id=4a0090947e4f447887c306cc2a565c1c, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.185808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.280873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 839/1672] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:39 2022] POST /identity/v3/projects => generated 371 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:39.284842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.296840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 834/1673] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:39 2022] PUT /identity/v3/projects/626826d727a94b1ca378c3bc99f8ab15/users/4a0090947e4f447887c306cc2a565c1c/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:39.301919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.494809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2cc15-f4ed-40b5-be53-f6fb33aeeb83 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.503649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.505661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.505661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.506993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.507811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.509971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.526649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 835/1674] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:00:39.538478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.545179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc47b0e-a8b8-4dec-83f1-3fa37f4ca2f6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.551648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3858e1d-420c-478c-b95a-532ac69ac1f1 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.557674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b5bc6-f286-4038-848b-febc443e4db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.558623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b5bc6-f286-4038-848b-febc443e4db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.559075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b5bc6-f286-4038-848b-febc443e4db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.559456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b5bc6-f286-4038-848b-febc443e4db6 admin admin] PATH_INFO: `/v3/projects/626826d727a94b1ca378c3bc99f8ab15/users/4a0090947e4f447887c306cc2a565c1c/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.562986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.563632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.563915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.564189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.564553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b5bc6-f286-4038-848b-febc443e4db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626826d727a94b1ca378c3bc99f8ab15, user_id=4a0090947e4f447887c306cc2a565c1c, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.564836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.567631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.571041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b5bc6-f286-4038-848b-febc443e4db6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.596100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 836/1675] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:39 2022] PUT /identity/v3/projects/626826d727a94b1ca378c3bc99f8ab15/users/4a0090947e4f447887c306cc2a565c1c/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:00:39.599422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.599779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.600747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.674366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] MFA Rules not processed for user `4a0090947e4f447887c306cc2a565c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:39.674366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:39.682420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 840/1676] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:39 2022] POST /identity/v3/users => generated 286 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:39.684029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.704037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ec42b-264b-4a9a-80bc-4090b835e244 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.821486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 837/1677] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:39.835836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.837407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:39.842636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.844359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.844643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.847660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.850044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.860073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.895021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 841/1678] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:39.898658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:39.930778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79bfe8-214b-4881-9fd2-03dd7b99c207 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.931223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56db87f-5044-4a24-8b45-ec24d419ccb9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:39.936145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.936884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.937206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.937488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.943226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:39.943881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:39.944134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:39.949406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:39.950512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.952187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.958300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:39.966271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:39.985735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 842/1679] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:39.989007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:40.001426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 838/1680] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:40.009761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97253cd8-c507-4863-bc1e-b0fc73778a83 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:40.020904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a8992-c659-4e6e-b6e9-ed1179b67d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:40.021659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a8992-c659-4e6e-b6e9-ed1179b67d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:40.022143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a8992-c659-4e6e-b6e9-ed1179b67d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:40.022437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a8992-c659-4e6e-b6e9-ed1179b67d81 admin admin] PATH_INFO: `/v3/projects/6f070626fcb447488cc42dbd27b71467/users/caf7ab33a52b4a75addc9f67dc0dc712/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:40.045529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a8992-c659-4e6e-b6e9-ed1179b67d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f070626fcb447488cc42dbd27b71467, user_id=caf7ab33a52b4a75addc9f67dc0dc712, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:40.050056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a8992-c659-4e6e-b6e9-ed1179b67d81 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:40.133540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:40.135263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:40.174692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 843/1681] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:39 2022] PUT /identity/v3/projects/6f070626fcb447488cc42dbd27b71467/users/caf7ab33a52b4a75addc9f67dc0dc712/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:40.182491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1743b53-6a81-4ff4-aacb-c3bfb308e4bb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:40.182991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1743b53-6a81-4ff4-aacb-c3bfb308e4bb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:40.184971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1743b53-6a81-4ff4-aacb-c3bfb308e4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:40.195512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddf158-f9fc-4332-a4d3-d1bf7d091728 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:40.244516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:40.266234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d1743b53-6a81-4ff4-aacb-c3bfb308e4bb None None] MFA Rules not processed for user `caf7ab33a52b4a75addc9f67dc0dc712`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:40.267376 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d1743b53-6a81-4ff4-aacb-c3bfb308e4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:40.366269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:40.367033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:40.367366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:40.367699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:40.386448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:40.439470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 844/1682] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:40.439470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1743b53-6a81-4ff4-aacb-c3bfb308e4bb None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:40.523563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:40.528634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:40.530400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:40.553721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 839/1683] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:00:40.656339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1743b53-6a81-4ff4-aacb-c3bfb308e4bb None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:40.668228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:40.668585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:40.668876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:40.669190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:40.671193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:40.673331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:40.891290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 845/1684] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:40 2022] POST /identity/v3/projects => generated 373 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:40.892695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:40.917239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7bb1-4d15-4f3d-9cf5-be1d41c0f86b service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:40.961607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:40.963787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:41.164908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:41.164908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:41.165594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:41.165594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:41.170833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:41.175848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:41.200881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b82b6c-066a-4672-8098-5705b0e0bc3f admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:41.211702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:41.212447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:41.212838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:41.213113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] PATH_INFO: `/v3/projects/69c0c03f491744169f699453bbb5ca86` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:41.217215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c0c03f491744169f699453bbb5ca86)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:41.219686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:41.532994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 840/1685] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:40 2022] POST /identity/v3/users => generated 293 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:41.532994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:41.532994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421d00-b209-4029-890a-857c68e4c7dc admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:41.532994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:41.534593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:41.534593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:41.534593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:41.534593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:41.534593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:41.534593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 841/1686] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:41.535333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:41.535333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b523a25-aba8-48bf-838b-e6fee5f89301 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:41.535333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:41.535333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:41.535850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:41.535850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] PATH_INFO: `/v3/projects/37351b55a4f54ae4b5843d3d1271fdf5/users/a29690e0b73e4551853ee847744428b5/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:41.535850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37351b55a4f54ae4b5843d3d1271fdf5, user_id=a29690e0b73e4551853ee847744428b5, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:41.535850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:41.617506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 842/1687] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:41 2022] PUT /identity/v3/projects/37351b55a4f54ae4b5843d3d1271fdf5/users/a29690e0b73e4551853ee847744428b5/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:41.621841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:41.635710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c0c03f491744169f699453bbb5ca86', 'request_id': 'req-d00dcaab-2208-4d32-9e60-9f03e1155446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:41.655476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c0c03f491744169f699453bbb5ca86', 'request_id': 'req-d00dcaab-2208-4d32-9e60-9f03e1155446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:41.677201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c0c03f491744169f699453bbb5ca86', 'request_id': 'req-d00dcaab-2208-4d32-9e60-9f03e1155446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:41.758220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5603d02-0119-4056-9f8f-1050940afdc6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:41.805747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] The token cache is being invalidate because project 69c0c03f491744169f699453bbb5ca86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:41.813417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:41.813417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d00dcaab-2208-4d32-9e60-9f03e1155446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:41.813417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 846/1688] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:40 2022] DELETE /identity/v3/projects/69c0c03f491744169f699453bbb5ca86 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:41.818927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764ecf-8ccc-4670-aa3e-5ed3fbb3c6e2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:41.819226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764ecf-8ccc-4670-aa3e-5ed3fbb3c6e2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:41.819507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764ecf-8ccc-4670-aa3e-5ed3fbb3c6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:41.902861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:41.914279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-49764ecf-8ccc-4670-aa3e-5ed3fbb3c6e2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:41.919242 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-49764ecf-8ccc-4670-aa3e-5ed3fbb3c6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:41.985352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:41.986021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:41.986305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:41.986583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:41.992723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:41.997003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:42.040953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 843/1689] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:42.042216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:42.045420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:42.081365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e565b67-f661-4914-a27d-c3645567b45d admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:42.095039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 847/1690] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:42.101656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ce980-280d-40ea-af1f-e98f07ccba20 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.108093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ce980-280d-40ea-af1f-e98f07ccba20 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.108093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ce980-280d-40ea-af1f-e98f07ccba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.108843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c533109-947d-4982-b60b-978f3c60e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:42.109607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c533109-947d-4982-b60b-978f3c60e439 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.109931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c533109-947d-4982-b60b-978f3c60e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.110291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c533109-947d-4982-b60b-978f3c60e439 admin admin] PATH_INFO: `/v3/projects/37351b55a4f54ae4b5843d3d1271fdf5/users/a29690e0b73e4551853ee847744428b5/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.117018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c533109-947d-4982-b60b-978f3c60e439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37351b55a4f54ae4b5843d3d1271fdf5, user_id=a29690e0b73e4551853ee847744428b5, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:42.127631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c533109-947d-4982-b60b-978f3c60e439 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:42.149323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2d3ce980-280d-40ea-af1f-e98f07ccba20 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:42.150752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2d3ce980-280d-40ea-af1f-e98f07ccba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:42.173912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 844/1691] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:42 2022] PUT /identity/v3/projects/37351b55a4f54ae4b5843d3d1271fdf5/users/a29690e0b73e4551853ee847744428b5/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:42.176550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5c7b-9b9e-4cd7-add5-177bf42a55d1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.177228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5c7b-9b9e-4cd7-add5-177bf42a55d1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.177678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5c7b-9b9e-4cd7-add5-177bf42a55d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.228080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e90e5c7b-9b9e-4cd7-add5-177bf42a55d1 None None] MFA Rules not processed for user `a29690e0b73e4551853ee847744428b5`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:42.230682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e90e5c7b-9b9e-4cd7-add5-177bf42a55d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:42.326083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 848/1692] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:42.328029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658e8d-c2b0-427a-b80c-44640d813cb7 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.328540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658e8d-c2b0-427a-b80c-44640d813cb7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.329103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658e8d-c2b0-427a-b80c-44640d813cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.397276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-7c658e8d-c2b0-427a-b80c-44640d813cb7 None None] MFA Rules not processed for user `41c61d68b9654b23a080d390f7768df9`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:42.401848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7c658e8d-c2b0-427a-b80c-44640d813cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:42.430699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 845/1693] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:42.431693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5c7b-9b9e-4cd7-add5-177bf42a55d1 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:42.452600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:42.461300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5c7b-9b9e-4cd7-add5-177bf42a55d1 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:42.472786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:42.473902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.474393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.475603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.476565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:42.477088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:42.481065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:42.524571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 846/1694] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:42.527283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:42.529331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:42.590223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 849/1695] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:42 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:42.591297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c658e8d-c2b0-427a-b80c-44640d813cb7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:42.610161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c658e8d-c2b0-427a-b80c-44640d813cb7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:42.626616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:42.627529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.627965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.628404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.629220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:42.631473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:42.729973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881911-0831-4b51-8228-b3136bf4c316 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:42.737186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:42.739290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.739697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.740388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.772082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:42.775198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:42.795598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 850/1696] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:42 2022] POST /identity/v3/projects => generated 363 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:42.796556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:42.807569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 847/1697] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:42.812247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:42.960215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f1cb-f10c-4854-8881-881361fcc2b6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:42.968072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:42.968700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:42.969099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:42.980956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:42.981725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:42.983759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:43.084398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:43.096042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a23f9-d5ee-4e75-a262-f58fa6021ef9 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:43.115240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 851/1698] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:42 2022] POST /identity/v3/users => generated 282 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:43.127572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:43.144759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d89d-81a4-4977-aabe-c530c5ef3334 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:43.174634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:43.175266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:43.175556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:43.175811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:43.177641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:43.184626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:43.218389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 852/1699] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:00:43.219089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:43.268316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373cf11e-439f-4dd7-82f4-e74b1fa02919 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:43.291587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:43.292158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:43.292518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:43.292876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:43.293522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:43.299920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2490b-7ad6-4877-857d-32e848b1f238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:43.300538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2490b-7ad6-4877-857d-32e848b1f238 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:43.300816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2490b-7ad6-4877-857d-32e848b1f238 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:43.301086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2490b-7ad6-4877-857d-32e848b1f238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:43.303371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:43.322184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fac2490b-7ad6-4877-857d-32e848b1f238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:43.350640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 853/1700] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:43.355040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:43.385629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604d39-511a-4727-b2b4-7adffc0cbc82 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:43.405378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:43.406174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:43.406578 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:43.408403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] PATH_INFO: `/v3/projects/8485b56b64b148ceb53ef79bd6c40478/users/c88a0755b6e74d318f3eb2073fb62106/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:43.432379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:43.465207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8485b56b64b148ceb53ef79bd6c40478, user_id=c88a0755b6e74d318f3eb2073fb62106, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:43.473329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:43.478077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2490b-7ad6-4877-857d-32e848b1f238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:43.479752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2490b-7ad6-4877-857d-32e848b1f238 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:43.500329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 848/1701] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:00:43.593121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 854/1702] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:43 2022] PUT /identity/v3/projects/8485b56b64b148ceb53ef79bd6c40478/users/c88a0755b6e74d318f3eb2073fb62106/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:43.605525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58af8-4ccd-4394-b6fd-e2c34caeab70 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:43.605888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58af8-4ccd-4394-b6fd-e2c34caeab70 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:43.606130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58af8-4ccd-4394-b6fd-e2c34caeab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:43.698908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0ef58af8-4ccd-4394-b6fd-e2c34caeab70 None None] MFA Rules not processed for user `c88a0755b6e74d318f3eb2073fb62106`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:43.700250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0ef58af8-4ccd-4394-b6fd-e2c34caeab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:43.949656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:44.003020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 849/1703] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:44.007782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:44.142713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84160-ee62-4c5d-84d9-44c3e3123478 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:44.155496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:44.156499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:44.156905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:44.157287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:44.158162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:44.161296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:44.348743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:44.366206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 855/1704] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:44 2022] POST /identity/v3/projects => generated 365 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:44.370343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58af8-4ccd-4394-b6fd-e2c34caeab70 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:44.393736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58af8-4ccd-4394-b6fd-e2c34caeab70 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:44.552435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:44.553537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:44.554002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:44.555190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:44.557704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:44.563711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:44.713451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 850/1705] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:44 2022] POST /identity/v3/users => generated 289 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:44.717728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:44.738831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432921e-1996-48a1-b42d-855cd3b4f4ae admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:44.755830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51375086-00b3-4410-8f19-03b76a3e5004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:44.756714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51375086-00b3-4410-8f19-03b76a3e5004 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:44.757065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51375086-00b3-4410-8f19-03b76a3e5004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:44.758871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51375086-00b3-4410-8f19-03b76a3e5004 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:44.760384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51375086-00b3-4410-8f19-03b76a3e5004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:44.766926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51375086-00b3-4410-8f19-03b76a3e5004 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:44.805928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 856/1706] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:44.807080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:44.822222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a62ea-1896-47ce-b7ec-3019ec5f848f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:44.840152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:44.841127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:44.841400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:44.841857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] PATH_INFO: `/v3/projects/6c583d8634eb4504a52b510695f4eff7/users/7af3df17bed84c06bd7d8c071eb6e371/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:44.884441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c583d8634eb4504a52b510695f4eff7, user_id=7af3df17bed84c06bd7d8c071eb6e371, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:44.893435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:44.901313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48f0c1-c47d-4459-b93a-61eecc5134b8 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:44.903005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48f0c1-c47d-4459-b93a-61eecc5134b8 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:44.903005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48f0c1-c47d-4459-b93a-61eecc5134b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:44.970128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-cd48f0c1-c47d-4459-b93a-61eecc5134b8 None None] MFA Rules not processed for user `a70246b8e8b842129c99827859b90478`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:44.972409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cd48f0c1-c47d-4459-b93a-61eecc5134b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:45.034296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 851/1707] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:44 2022] PUT /identity/v3/projects/6c583d8634eb4504a52b510695f4eff7/users/7af3df17bed84c06bd7d8c071eb6e371/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:45.035692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:45.140944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 857/1708] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:45.142641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7154cf6-9894-4de4-a15e-5bf318953bc3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:45.142933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7154cf6-9894-4de4-a15e-5bf318953bc3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:45.143212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7154cf6-9894-4de4-a15e-5bf318953bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:45.188143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636ee12-0337-4479-bf29-b31fc1d1c69e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:45.194119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b7154cf6-9894-4de4-a15e-5bf318953bc3 None None] MFA Rules not processed for user `a70246b8e8b842129c99827859b90478`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:45.201305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b7154cf6-9894-4de4-a15e-5bf318953bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:45.207689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64dd5f-429f-4c78-8c3d-e207f1b104ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:45.208358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64dd5f-429f-4c78-8c3d-e207f1b104ab admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:45.208685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64dd5f-429f-4c78-8c3d-e207f1b104ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:45.212401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64dd5f-429f-4c78-8c3d-e207f1b104ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:45.213142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d64dd5f-429f-4c78-8c3d-e207f1b104ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:45.221102 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d64dd5f-429f-4c78-8c3d-e207f1b104ab admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:45.271482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 852/1709] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:45.280881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:45.281720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:45.281720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:45.366922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] MFA Rules not processed for user `1943057b69d342d9b45fc02be9f3d5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:45.373520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:45.521340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 858/1710] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:45.529846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7154cf6-9894-4de4-a15e-5bf318953bc3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:45.578799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7154cf6-9894-4de4-a15e-5bf318953bc3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:45.599573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:45.600425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:45.607128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:45.607128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] PATH_INFO: `/v3/projects/6c583d8634eb4504a52b510695f4eff7/users/7af3df17bed84c06bd7d8c071eb6e371/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:45.611375 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c583d8634eb4504a52b510695f4eff7, user_id=7af3df17bed84c06bd7d8c071eb6e371, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:45.614185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:45.623001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 853/1711] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:45.626242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:45.630418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:45.688290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 859/1712] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:45 2022] PUT /identity/v3/projects/6c583d8634eb4504a52b510695f4eff7/users/7af3df17bed84c06bd7d8c071eb6e371/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:45.699429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:45.701112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:45.859457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4523dc-8df5-4cda-b92b-517827f1085c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:45.996177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:46.017300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9c840-58f0-4576-bdc6-af0e9df4a8eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:46.019766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9c840-58f0-4576-bdc6-af0e9df4a8eb service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:46.024076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9c840-58f0-4576-bdc6-af0e9df4a8eb service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:46.024358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9c840-58f0-4576-bdc6-af0e9df4a8eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:46.108824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa6867-8b1e-45c8-9b8e-d2f8b26a826a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:46.132785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:46.133874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:46.134332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:46.135326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:46.244114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9c840-58f0-4576-bdc6-af0e9df4a8eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:46.246936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9c840-58f0-4576-bdc6-af0e9df4a8eb service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:46.289680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:46.290651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 854/1713] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:46.292409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70101-20b2-4c4f-bf0d-556d240d3664 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:46.292846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70101-20b2-4c4f-bf0d-556d240d3664 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:46.294419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70101-20b2-4c4f-bf0d-556d240d3664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:46.307065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:46.409635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 860/1714] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:46.422906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:46.439541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4ff70101-20b2-4c4f-bf0d-556d240d3664 None None] MFA Rules not processed for user `7af3df17bed84c06bd7d8c071eb6e371`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:46.448010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4ff70101-20b2-4c4f-bf0d-556d240d3664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:46.613750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:46.744484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 855/1715] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:46.756799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:46.759470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:46.943125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4def7e-3fef-4831-80df-04b4664c9e7e service nova] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:46.957181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113bceb-d6e0-431c-85a6-e26c60d3ce5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:46.958307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113bceb-d6e0-431c-85a6-e26c60d3ce5e service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:46.958789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113bceb-d6e0-431c-85a6-e26c60d3ce5e service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:46.960194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113bceb-d6e0-431c-85a6-e26c60d3ce5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:47.007050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113bceb-d6e0-431c-85a6-e26c60d3ce5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:47.011773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113bceb-d6e0-431c-85a6-e26c60d3ce5e service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:47.036814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 861/1716] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:47.442371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18533203-ad8c-435c-98b4-400a4c4e0c8f None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:47.442918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18533203-ad8c-435c-98b4-400a4c4e0c8f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:47.443007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18533203-ad8c-435c-98b4-400a4c4e0c8f None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:47.447821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 856/1717] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:00:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:47.477212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e9501-7c4a-4db6-ac29-90c7197fc1e9 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:47.477707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e9501-7c4a-4db6-ac29-90c7197fc1e9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:47.477890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e9501-7c4a-4db6-ac29-90c7197fc1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:47.547728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-265e9501-7c4a-4db6-ac29-90c7197fc1e9 None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:47.549810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-265e9501-7c4a-4db6-ac29-90c7197fc1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:47.766750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:47.790040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 862/1718] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:00:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:47.877721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18533203-ad8c-435c-98b4-400a4c4e0c8f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:47.887101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:47.888142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-18533203-ad8c-435c-98b4-400a4c4e0c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:47.941957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18533203-ad8c-435c-98b4-400a4c4e0c8f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:47.952303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:47.953294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:47.954061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:47.956469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:47.987664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:47.989689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:48.016170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 857/1719] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:48.097655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f16e38-989d-445e-a5fe-444565c71bff None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:48.098416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f16e38-989d-445e-a5fe-444565c71bff None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:48.099081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f16e38-989d-445e-a5fe-444565c71bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:48.168792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b5f16e38-989d-445e-a5fe-444565c71bff None None] MFA Rules not processed for user `c0afa15991f14e0ca0d3701d06a89049`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:48.170252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b5f16e38-989d-445e-a5fe-444565c71bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:48.302063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 863/1720] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:48.320548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:48.405032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52535e3-da93-4c2d-9016-0d9843b8dc72 service placement] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:48.448381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:48.449092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:48.449432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:48.449733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:48.502397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbab1af-6c8f-40da-9dc3-49cc84678ef9 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:48.503295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbab1af-6c8f-40da-9dc3-49cc84678ef9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:48.503721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbab1af-6c8f-40da-9dc3-49cc84678ef9 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:48.504651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:48.506446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 864/1721] 10.222.0.61 () {54 vars in 912 bytes} [Thu Sep 1 14:00:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:48.511588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:48.518404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:48.518404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:48.518404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:48.550316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 858/1722] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:48.575961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:48.577802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:48.649138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] MFA Rules not processed for user `b1c427a987ba48878873734ecf0f86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:48.660231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:48.790919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 865/1723] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:00:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:48.812977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:48.868865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a962c-d2e8-4f9e-b553-0a4f948825c1 service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:48.881583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:48.882448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:48.882732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:48.883009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] PATH_INFO: `/v3/users/5afb1b8bdba54493bd9e56d899798c56` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:48.885730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5afb1b8bdba54493bd9e56d899798c56)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:48.889272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:48.926077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:48.927582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:48.942029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257894-8828-4b73-85b5-4a2cc3ff7445 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:48.946767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:48.947655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:48.948122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:48.949280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:48.960387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:48.962175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:48.973937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 866/1724] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:48.974827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:48.989282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429eb7-84af-47de-b0b2-48d4cf45a415 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:49.001536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:49.002626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:49.002865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:49.003132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] PATH_INFO: `/v3/users/c88a0755b6e74d318f3eb2073fb62106` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:49.008015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c88a0755b6e74d318f3eb2073fb62106)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:49.010642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:49.188808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5afb1b8bdba54493bd9e56d899798c56'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.204105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5afb1b8bdba54493bd9e56d899798c56'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.319693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5afb1b8bdba54493bd9e56d899798c56'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.320359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5afb1b8bdba54493bd9e56d899798c56'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.350035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:49.352636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-538ce6cc-912b-4107-a396-8ea87ddf4789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5afb1b8bdba54493bd9e56d899798c56'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.377382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c88a0755b6e74d318f3eb2073fb62106'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.380650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 859/1725] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:48 2022] DELETE /identity/v3/users/5afb1b8bdba54493bd9e56d899798c56 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:00:49.388651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34f24-2252-4137-ac06-11c52dbc0f1c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:49.390964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34f24-2252-4137-ac06-11c52dbc0f1c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:49.391740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34f24-2252-4137-ac06-11c52dbc0f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:49.398038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c88a0755b6e74d318f3eb2073fb62106'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.400420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c88a0755b6e74d318f3eb2073fb62106'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.402120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c88a0755b6e74d318f3eb2073fb62106'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.434622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-24f34f24-2252-4137-ac06-11c52dbc0f1c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:49.441017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-24f34f24-2252-4137-ac06-11c52dbc0f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:49.543136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c88a0755b6e74d318f3eb2073fb62106'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:49.547204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 860/1726] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:49.548633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99362cd-1252-4783-bf06-f0e506c656a1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:49.560740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99362cd-1252-4783-bf06-f0e506c656a1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:49.560740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99362cd-1252-4783-bf06-f0e506c656a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:49.592368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 867/1727] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:48 2022] DELETE /identity/v3/users/c88a0755b6e74d318f3eb2073fb62106 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:49.598526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:49.606790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:49.656211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e99362cd-1252-4783-bf06-f0e506c656a1 None None] MFA Rules not processed for user `b77f38582b8842d7a303e8431299191a`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:49.657554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e99362cd-1252-4783-bf06-f0e506c656a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:49.743416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47978d4d-ccf5-4fad-8e71-98e00bb4749b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:49.750952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:49.751604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:49.751880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:49.752157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:49.851162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:49.872509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 861/1728] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:49.873866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:49.874161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:49.874437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:49.927034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:49.935232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:49.942391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:49.945262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:49.961892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 862/1729] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:49.962711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:49.967962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:49.986233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 868/1730] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:49.989522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:50.022209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50106087-386e-4a8c-a76b-09a79a5463d2 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:50.043881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:50.044531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:50.044817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:50.045075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:50.083116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:50.085045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:50.128449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 869/1731] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:50.154625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819623e-2e3c-448a-b8c3-184b17004b4e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:50.171264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:50.172358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:50.172854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:50.173389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:50.192803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:50.197757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:50.235217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 863/1732] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:50.426374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:50.428720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:50.552965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:50.555477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:50.583221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062f929-30aa-43b4-a471-d1df404731ef service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:50.602522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:50.602522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:50.602522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:50.602522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] PATH_INFO: `/v3/users/1bf9aa4f1cc0435e9983244256a2374b` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:50.606755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf9aa4f1cc0435e9983244256a2374b)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:50.607415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71487056-cbc6-4e17-958a-58f3748d40be service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:50.626131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:50.632051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:50.632051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:50.632051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:50.632386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:50.633425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:50.668613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:50.670352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:50.683246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 870/1733] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:50.944367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf9aa4f1cc0435e9983244256a2374b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:50.962781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf9aa4f1cc0435e9983244256a2374b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:51.060441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf9aa4f1cc0435e9983244256a2374b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:51.061512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf9aa4f1cc0435e9983244256a2374b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:51.123455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf9aa4f1cc0435e9983244256a2374b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:51.141413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 864/1734] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:50 2022] DELETE /identity/v3/users/1bf9aa4f1cc0435e9983244256a2374b => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:51.285254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:51.483783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:51.485464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:51.514405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfda5c-2fa7-4a7e-a08e-0adcb3b247db service placement] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:51.608643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:51.610386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:51.706790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:51.713307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:51.716139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:51.716650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] PATH_INFO: `/v3/projects/49f8266d74614e14ae10be9b90535ff9` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:51.719700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f8266d74614e14ae10be9b90535ff9)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:51.723540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:51.810472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63760f-86cf-4bae-9d41-91b243c888be admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:51.815778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:51.816459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:51.816740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:51.817019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] PATH_INFO: `/v3/projects/8485b56b64b148ceb53ef79bd6c40478` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:51.821491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8485b56b64b148ceb53ef79bd6c40478)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:51.823804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:51.944784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f8266d74614e14ae10be9b90535ff9', 'request_id': 'req-36137ff3-b25a-4369-bcf8-f773031105e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:51.962798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f8266d74614e14ae10be9b90535ff9', 'request_id': 'req-36137ff3-b25a-4369-bcf8-f773031105e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:51.992083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f8266d74614e14ae10be9b90535ff9', 'request_id': 'req-36137ff3-b25a-4369-bcf8-f773031105e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.011554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8485b56b64b148ceb53ef79bd6c40478', 'request_id': 'req-915ecea5-3a33-41b7-b905-da9924522155'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.019178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8485b56b64b148ceb53ef79bd6c40478', 'request_id': 'req-915ecea5-3a33-41b7-b905-da9924522155'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.177064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] The token cache is being invalidate because project 49f8266d74614e14ae10be9b90535ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:52.177228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.178485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.181802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 871/1735] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:51 2022] DELETE /identity/v3/projects/49f8266d74614e14ae10be9b90535ff9 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:52.186964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:52.188956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:52.296039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8485b56b64b148ceb53ef79bd6c40478', 'request_id': 'req-915ecea5-3a33-41b7-b905-da9924522155'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.349525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36137ff3-b25a-4369-bcf8-f773031105e7 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:52.350182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] The token cache is being invalidate because project 8485b56b64b148ceb53ef79bd6c40478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:52.351248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.358633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.371767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 865/1736] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:51 2022] DELETE /identity/v3/projects/8485b56b64b148ceb53ef79bd6c40478 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:52.374525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:52.379754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:52.447663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:52.451971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:52.452424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:52.453231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:52.453672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] PATH_INFO: `/v3/users/4a0090947e4f447887c306cc2a565c1c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:52.455225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0090947e4f447887c306cc2a565c1c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:52.460527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:52.621601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ecea5-3a33-41b7-b905-da9924522155 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:52.631211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:52.632408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:52.632577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:52.633050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] PATH_INFO: `/v3/projects/048ede8fff72416da06a3a99f9ed9370` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:52.635036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] RBAC: Authorizing `identity:delete_project(project_id=048ede8fff72416da06a3a99f9ed9370)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:52.639739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:52.734191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0090947e4f447887c306cc2a565c1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.743799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0090947e4f447887c306cc2a565c1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.744595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0090947e4f447887c306cc2a565c1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.745888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0090947e4f447887c306cc2a565c1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.845550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048ede8fff72416da06a3a99f9ed9370', 'request_id': 'req-19cd00b2-97d4-4168-bc87-7d071534033e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.853641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048ede8fff72416da06a3a99f9ed9370', 'request_id': 'req-19cd00b2-97d4-4168-bc87-7d071534033e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.873117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0090947e4f447887c306cc2a565c1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.920607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 872/1737] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:52 2022] DELETE /identity/v3/users/4a0090947e4f447887c306cc2a565c1c => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:52.921751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:52.930296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:52.965147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048ede8fff72416da06a3a99f9ed9370', 'request_id': 'req-19cd00b2-97d4-4168-bc87-7d071534033e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.989569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] The token cache is being invalidate because project 048ede8fff72416da06a3a99f9ed9370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:52.989938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:52.994215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:52.995776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.005808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 866/1738] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:52 2022] DELETE /identity/v3/projects/048ede8fff72416da06a3a99f9ed9370 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:53.010085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:53.016753 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:53.120974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79773d-918c-4069-9416-cadf049a5989 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:53.129642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:53.130765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:53.131253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:53.131723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] PATH_INFO: `/v3/users/7af3df17bed84c06bd7d8c071eb6e371` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:53.134253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af3df17bed84c06bd7d8c071eb6e371)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:53.140444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:53.216809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd00b2-97d4-4168-bc87-7d071534033e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:53.223832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:53.224535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:53.224817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:53.225093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] PATH_INFO: `/v3/users/41c61d68b9654b23a080d390f7768df9` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:53.227169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c61d68b9654b23a080d390f7768df9)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:53.229244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:53.480614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af3df17bed84c06bd7d8c071eb6e371'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.498769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af3df17bed84c06bd7d8c071eb6e371'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.503331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af3df17bed84c06bd7d8c071eb6e371'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.504416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af3df17bed84c06bd7d8c071eb6e371'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.601124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c61d68b9654b23a080d390f7768df9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.611845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c61d68b9654b23a080d390f7768df9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.615537 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af3df17bed84c06bd7d8c071eb6e371'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.648506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 873/1739] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:52 2022] DELETE /identity/v3/users/7af3df17bed84c06bd7d8c071eb6e371 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:53.706487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c61d68b9654b23a080d390f7768df9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.707305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c61d68b9654b23a080d390f7768df9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.738473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c355e495-9f8d-4162-ab08-a2ec8f5ba231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c61d68b9654b23a080d390f7768df9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:53.755743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 867/1740] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:53 2022] DELETE /identity/v3/users/41c61d68b9654b23a080d390f7768df9 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:54.427396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:54.429088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:54.609119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6740a-4970-4961-b779-81d1a57c8f71 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:54.688144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:54.688812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:54.689098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:54.689428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] PATH_INFO: `/v3/projects/626826d727a94b1ca378c3bc99f8ab15` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:54.690916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=626826d727a94b1ca378c3bc99f8ab15)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:54.693801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:54.736295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:54.736874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:54.737181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:54.762163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] MFA Rules not processed for user `1943057b69d342d9b45fc02be9f3d5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:54.763897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:54.844008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 868/1741] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:00:54.845574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:54.846517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626826d727a94b1ca378c3bc99f8ab15', 'request_id': 'req-4660abbf-89aa-4a55-b599-435285876b1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:54.848689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:54.864410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626826d727a94b1ca378c3bc99f8ab15', 'request_id': 'req-4660abbf-89aa-4a55-b599-435285876b1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:54.871794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626826d727a94b1ca378c3bc99f8ab15', 'request_id': 'req-4660abbf-89aa-4a55-b599-435285876b1c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:54.874111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a259a1-2cea-4a83-aad9-24fc88014f63 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:54.992438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:54.993354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:54.993881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:54.994366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] PATH_INFO: `/v3/projects/6c583d8634eb4504a52b510695f4eff7` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:54.996095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c583d8634eb4504a52b510695f4eff7)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:54.999039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:55.007600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] The token cache is being invalidate because project 626826d727a94b1ca378c3bc99f8ab15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:55.007796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.010199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4660abbf-89aa-4a55-b599-435285876b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.031680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 874/1742] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:54 2022] DELETE /identity/v3/projects/626826d727a94b1ca378c3bc99f8ab15 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:55.033690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.034026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.034291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.106847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:55.106847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.128768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 875/1743] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:55.143133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:55.143133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.146922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ce82e-0ae6-4554-804d-6d4dd2672596 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:55.151992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:55.152637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.152917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.153194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] PATH_INFO: `/v3/projects/81d15464f866479486c3b593e9440a82` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.154637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d15464f866479486c3b593e9440a82)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:55.157460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:55.274350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c583d8634eb4504a52b510695f4eff7', 'request_id': 'req-d76bad03-102a-4951-9411-1259c2327f86'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.289664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c583d8634eb4504a52b510695f4eff7', 'request_id': 'req-d76bad03-102a-4951-9411-1259c2327f86'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.369462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c583d8634eb4504a52b510695f4eff7', 'request_id': 'req-d76bad03-102a-4951-9411-1259c2327f86'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.389980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d15464f866479486c3b593e9440a82', 'request_id': 'req-6ff8a51b-0c09-4543-a3f4-6f8603d77413'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.391428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] The token cache is being invalidate because project 6c583d8634eb4504a52b510695f4eff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:55.391890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.394285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d76bad03-102a-4951-9411-1259c2327f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.399590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 869/1744] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:54 2022] DELETE /identity/v3/projects/6c583d8634eb4504a52b510695f4eff7 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:55.401972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d55493-3848-4078-b9c4-eac840f4ebba None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.404353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d55493-3848-4078-b9c4-eac840f4ebba None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.409491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d55493-3848-4078-b9c4-eac840f4ebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.409645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d15464f866479486c3b593e9440a82', 'request_id': 'req-6ff8a51b-0c09-4543-a3f4-6f8603d77413'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.421321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d15464f866479486c3b593e9440a82', 'request_id': 'req-6ff8a51b-0c09-4543-a3f4-6f8603d77413'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.455705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-25d55493-3848-4078-b9c4-eac840f4ebba None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:55.456971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-25d55493-3848-4078-b9c4-eac840f4ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.474325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:55.545484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] The token cache is being invalidate because project 81d15464f866479486c3b593e9440a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:00:55.546416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.547541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6ff8a51b-0c09-4543-a3f4-6f8603d77413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:00:55.552913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 876/1745] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:00:55 2022] DELETE /identity/v3/projects/81d15464f866479486c3b593e9440a82 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:55.557206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651ecbb-cea3-4367-816e-12e847a2d71d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.558097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651ecbb-cea3-4367-816e-12e847a2d71d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.558883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651ecbb-cea3-4367-816e-12e847a2d71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.590000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 870/1746] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:55.592155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34404adc-de26-41f4-8dff-5c42b30eee20 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.592614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34404adc-de26-41f4-8dff-5c42b30eee20 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.593927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34404adc-de26-41f4-8dff-5c42b30eee20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.620548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-34404adc-de26-41f4-8dff-5c42b30eee20 None None] MFA Rules not processed for user `c0afa15991f14e0ca0d3701d06a89049`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:55.623560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-34404adc-de26-41f4-8dff-5c42b30eee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.644947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6651ecbb-cea3-4367-816e-12e847a2d71d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:55.646871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6651ecbb-cea3-4367-816e-12e847a2d71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.668788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 877/1747] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:55.670494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e353-3fe6-4cee-aa5c-79ed463227e3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.670967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e353-3fe6-4cee-aa5c-79ed463227e3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.671562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e353-3fe6-4cee-aa5c-79ed463227e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.708865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f467e353-3fe6-4cee-aa5c-79ed463227e3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:55.710857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f467e353-3fe6-4cee-aa5c-79ed463227e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.739072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 878/1748] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:55.754057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e353-3fe6-4cee-aa5c-79ed463227e3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:55.785178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 871/1749] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:55.785178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a655512-fcf9-48d7-96fc-10fce293cba0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.785178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a655512-fcf9-48d7-96fc-10fce293cba0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.785178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a655512-fcf9-48d7-96fc-10fce293cba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.793618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e353-3fe6-4cee-aa5c-79ed463227e3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:55.805647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3c3ae-67fb-4905-921c-f615d9305f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:55.807005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3c3ae-67fb-4905-921c-f615d9305f4e admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.807288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3c3ae-67fb-4905-921c-f615d9305f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.808088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3c3ae-67fb-4905-921c-f615d9305f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.808812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3c3ae-67fb-4905-921c-f615d9305f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:55.809116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3c3ae-67fb-4905-921c-f615d9305f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:55.810927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3c3ae-67fb-4905-921c-f615d9305f4e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:55.827605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 879/1750] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:55.828791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-5a655512-fcf9-48d7-96fc-10fce293cba0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:55.830052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5a655512-fcf9-48d7-96fc-10fce293cba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.834275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4d9d-99c1-4fd3-8679-78f398dc3a04 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.834573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4d9d-99c1-4fd3-8679-78f398dc3a04 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.834888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4d9d-99c1-4fd3-8679-78f398dc3a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.864832 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-645b4d9d-99c1-4fd3-8679-78f398dc3a04 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:55.867207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-645b4d9d-99c1-4fd3-8679-78f398dc3a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:55.884730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 872/1751] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:55.884730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a655512-fcf9-48d7-96fc-10fce293cba0 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:55.885076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 880/1752] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:55.886203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4d9d-99c1-4fd3-8679-78f398dc3a04 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:55.905456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a655512-fcf9-48d7-96fc-10fce293cba0 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:55.910636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4d9d-99c1-4fd3-8679-78f398dc3a04 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:55.917965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:55.918782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.919741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.921354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.922167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:55.922473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:55.922753 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:55.923173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.923506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.923781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.924431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:55.924728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:55.932894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:55.945118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 881/1753] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:55.947256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:55.961341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1a57c-fb83-469e-adfc-28a801a5af04 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:55.988693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:55.989819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:55.990126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:55.991149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:55.991849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:00:55.992156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:56.001202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:56.012109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:56.036888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 882/1754] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:00:56.037664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:56.062768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7984ece-70ca-4cf2-a1e2-2c7b442ce280 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:56.069812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:56.071375 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:56.071651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:56.071918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:56.072574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:56.075624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:56.091911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 873/1755] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:55 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:56.093297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:56.238212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0c6a1-18c3-482a-aad2-36e3baacdcfb admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:56.251710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:56.252436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:56.252788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:56.253140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:56.253961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:56.256122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:56.268851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 883/1756] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:56 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:56.270713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:56.379940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 874/1757] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:56 2022] POST /identity/v3/projects => generated 355 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:56.381131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:56.525070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80487ec-3103-4288-ad38-6e06aa4fd83f admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:56.567556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39e887-6527-4a0c-a95c-98a1d48e045e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:56.583854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:56.584690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:56.584979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:56.585253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:56.585943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:56.588168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:56.665584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b758f6-0922-4c2b-a26b-3beabc375d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:56.666313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b758f6-0922-4c2b-a26b-3beabc375d80 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:56.666506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b758f6-0922-4c2b-a26b-3beabc375d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:56.667320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b758f6-0922-4c2b-a26b-3beabc375d80 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:56.668092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b758f6-0922-4c2b-a26b-3beabc375d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:56.670132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b758f6-0922-4c2b-a26b-3beabc375d80 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:56.695552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 875/1758] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:56 2022] POST /identity/v3/users => generated 281 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:56.695552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:56.707429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84925b49-7cac-4fd2-9fcd-e3e87650391f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:56.712524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:56.713753 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:56.714065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:56.714488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:56.715145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:56.717162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:56.804063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 884/1759] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:56 2022] POST /identity/v3/users => generated 286 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:56.804063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efea31-d95b-45ad-90dd-86a40dd5b5d5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:56.804063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efea31-d95b-45ad-90dd-86a40dd5b5d5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:56.804063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efea31-d95b-45ad-90dd-86a40dd5b5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:56.837325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 876/1760] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:56 2022] POST /identity/v3/users => generated 278 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:56.837990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:56.857799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048bfdb-720f-4947-abe3-a0bf76321bff admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:56.897425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:56.898094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:56.898381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:56.898660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:56.899339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:56.904231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:56.948807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 877/1761] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:56.950734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:56.958873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-05efea31-d95b-45ad-90dd-86a40dd5b5d5 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:56.966188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-05efea31-d95b-45ad-90dd-86a40dd5b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:56.986073 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee526f-9950-4f06-a28f-3f4270955f31 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:56.999143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:56.999780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.000124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.000404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.001068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.003673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.023397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 878/1762] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:57.024305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.068839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea94fbd-56c2-4a94-8d39-b28d34f942ba admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:57.082576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb8dcd-6e82-44c9-bc55-2e76d6221922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:57.083510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb8dcd-6e82-44c9-bc55-2e76d6221922 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.083955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb8dcd-6e82-44c9-bc55-2e76d6221922 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.084384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb8dcd-6e82-44c9-bc55-2e76d6221922 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.085214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb8dcd-6e82-44c9-bc55-2e76d6221922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.087820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb8dcd-6e82-44c9-bc55-2e76d6221922 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.128474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 879/1763] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:57.130902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99155-4a8b-4a3e-af54-3156a0dcf271 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.131192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99155-4a8b-4a3e-af54-3156a0dcf271 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.131556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99155-4a8b-4a3e-af54-3156a0dcf271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.162921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 885/1764] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:00:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:57.163990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05efea31-d95b-45ad-90dd-86a40dd5b5d5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.183152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ffc99155-4a8b-4a3e-af54-3156a0dcf271 None None] MFA Rules not processed for user `b77f38582b8842d7a303e8431299191a`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:57.185586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ffc99155-4a8b-4a3e-af54-3156a0dcf271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:57.190829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05efea31-d95b-45ad-90dd-86a40dd5b5d5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:57.196467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f514cd-cb57-482b-8e2f-7220221a3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:57.197115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f514cd-cb57-482b-8e2f-7220221a3115 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.198226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f514cd-cb57-482b-8e2f-7220221a3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.198511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f514cd-cb57-482b-8e2f-7220221a3115 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.200861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f514cd-cb57-482b-8e2f-7220221a3115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.207219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f514cd-cb57-482b-8e2f-7220221a3115 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.226566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 886/1765] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:57.238132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87589b1b-6741-4dc9-8c26-ad9198d9b6ac None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.238410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87589b1b-6741-4dc9-8c26-ad9198d9b6ac None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.238691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87589b1b-6741-4dc9-8c26-ad9198d9b6ac None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.244325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 887/1766] 10.222.0.61 () {54 vars in 912 bytes} [Thu Sep 1 14:00:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:00:57.246059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87589b1b-6741-4dc9-8c26-ad9198d9b6ac None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.264372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87589b1b-6741-4dc9-8c26-ad9198d9b6ac None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:57.269616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:57.270276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.270591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.270867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.271674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.279677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.290379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 888/1767] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:57.291498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.301120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 880/1768] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:57.302257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc99155-4a8b-4a3e-af54-3156a0dcf271 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.313870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e202a6-d33e-4ccf-9aae-7886aa56266c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:57.319933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376a8b7-cce8-4a2c-9afd-9ab556e464c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:57.320280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a8b7-cce8-4a2c-9afd-9ab556e464c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.326489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a8b7-cce8-4a2c-9afd-9ab556e464c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.326489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a8b7-cce8-4a2c-9afd-9ab556e464c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.326489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376a8b7-cce8-4a2c-9afd-9ab556e464c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.326489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376a8b7-cce8-4a2c-9afd-9ab556e464c4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.326879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc99155-4a8b-4a3e-af54-3156a0dcf271 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:57.344040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:57.344040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.344040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.344040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] PATH_INFO: `/v3/projects/fb6fce1a3693467c92e6fcb20e053bf3/users/1949477a74a34380bc106dedb6d5cc2a/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.349470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 889/1769] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:00:57.349470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdb54a-25db-47dc-864e-0c95d5e7419b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.349470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdb54a-25db-47dc-864e-0c95d5e7419b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.349470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdb54a-25db-47dc-864e-0c95d5e7419b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.382880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-14cdb54a-25db-47dc-864e-0c95d5e7419b None None] MFA Rules not processed for user `b1c427a987ba48878873734ecf0f86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:57.386773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-14cdb54a-25db-47dc-864e-0c95d5e7419b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:57.401270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6fce1a3693467c92e6fcb20e053bf3, user_id=1949477a74a34380bc106dedb6d5cc2a, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.403768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.549868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 890/1770] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:00:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:57.551021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdb54a-25db-47dc-864e-0c95d5e7419b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.554369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 881/1771] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:57 2022] PUT /identity/v3/projects/fb6fce1a3693467c92e6fcb20e053bf3/users/1949477a74a34380bc106dedb6d5cc2a/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:57.555291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.582835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdb54a-25db-47dc-864e-0c95d5e7419b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:57.708465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd888b-2d4a-4209-ab88-1a8db6207c78 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:57.714502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:57.715222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.715481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.715924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] PATH_INFO: `/v3/projects/b35b0a25114d4145a076f0ed81c2d0b8/users/5cf40efb96ce47cf95cfbca7faabce43/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.735818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b938721-b53f-47de-a69e-fed367fd9170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:57.738503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b938721-b53f-47de-a69e-fed367fd9170 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.741933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b938721-b53f-47de-a69e-fed367fd9170 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.742231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b938721-b53f-47de-a69e-fed367fd9170 admin admin] PATH_INFO: `/v3/projects/ba6c04abb9264f44aa904fcaf9732fd0/users/205a8a8551604044ac81b836f1c301d7/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:57.756960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35b0a25114d4145a076f0ed81c2d0b8, user_id=5cf40efb96ce47cf95cfbca7faabce43, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.763532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.793893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b938721-b53f-47de-a69e-fed367fd9170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6c04abb9264f44aa904fcaf9732fd0, user_id=205a8a8551604044ac81b836f1c301d7, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:57.799317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b938721-b53f-47de-a69e-fed367fd9170 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:57.869458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 882/1772] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:57 2022] PUT /identity/v3/projects/b35b0a25114d4145a076f0ed81c2d0b8/users/5cf40efb96ce47cf95cfbca7faabce43/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:57.873115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:57.879011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:57.936686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 891/1773] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:00:57 2022] PUT /identity/v3/projects/ba6c04abb9264f44aa904fcaf9732fd0/users/205a8a8551604044ac81b836f1c301d7/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:00:57.936686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ecb91-d506-466c-a84e-a49f493ac3d0 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:57.936686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ecb91-d506-466c-a84e-a49f493ac3d0 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:57.936686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ecb91-d506-466c-a84e-a49f493ac3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:58.002945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e133863-f061-48d3-b7e5-ded34608ba0d admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:58.034371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-db1ecb91-d506-466c-a84e-a49f493ac3d0 None None] MFA Rules not processed for user `1949477a74a34380bc106dedb6d5cc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:58.038278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-db1ecb91-d506-466c-a84e-a49f493ac3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:58.089914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:58.137710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:58.138399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:58.138859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:58.139308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:58.154541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:58.157106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:58.171165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 883/1774] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:58.179660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:58.183057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:58.195890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 892/1775] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:58.199571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b08a1-20b1-4b24-a196-8c6ef61ef276 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:58.200112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b08a1-20b1-4b24-a196-8c6ef61ef276 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:58.200522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b08a1-20b1-4b24-a196-8c6ef61ef276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:58.258740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-490b08a1-20b1-4b24-a196-8c6ef61ef276 None None] MFA Rules not processed for user `5cf40efb96ce47cf95cfbca7faabce43`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:58.260845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-490b08a1-20b1-4b24-a196-8c6ef61ef276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:58.388793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 893/1776] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:58.391418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7833-ec70-4eb6-9103-0a63a84f8fa3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:58.391908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7833-ec70-4eb6-9103-0a63a84f8fa3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:58.392099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7833-ec70-4eb6-9103-0a63a84f8fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:58.438832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:00:58.443928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55be229-e5dd-41e3-be04-07d30b836ba6 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:58.451450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-ddeb7833-ec70-4eb6-9103-0a63a84f8fa3 None None] MFA Rules not processed for user `205a8a8551604044ac81b836f1c301d7`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:58.452725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ddeb7833-ec70-4eb6-9103-0a63a84f8fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:58.460621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:58.462662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:58.476402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:58.476928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:58.616226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 894/1777] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:00:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:58.617601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb7833-ec70-4eb6-9103-0a63a84f8fa3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:58.662059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:58.664041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:58.696746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 884/1778] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:00:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:00:58.705023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:58.785387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb7833-ec70-4eb6-9103-0a63a84f8fa3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:58.797980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:58.800028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:58.801483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:58.801796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:58.803268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:58.806937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:58.913457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca3758-7282-4b03-b02a-60a0131e0211 service swift] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:58.922044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f30fa-c9a4-4363-8e9a-b984f4a82781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:58.922718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f30fa-c9a4-4363-8e9a-b984f4a82781 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:58.927785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f30fa-c9a4-4363-8e9a-b984f4a82781 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:58.928120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f30fa-c9a4-4363-8e9a-b984f4a82781 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:58.928827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f30fa-c9a4-4363-8e9a-b984f4a82781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:58.930839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f30fa-c9a4-4363-8e9a-b984f4a82781 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:59.043413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 895/1779] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:58 2022] POST /identity/v3/projects => generated 363 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:59.044632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:59.051076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:59.184560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e83cb-c8dc-468e-8f65-2f8eb72728d1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:59.184705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 885/1780] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:00:58 2022] POST /identity/v3/projects => generated 353 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:59.186698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be52c77-eb7b-4a9a-823b-9ede1a846176 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:59.187165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be52c77-eb7b-4a9a-823b-9ede1a846176 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:59.187587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be52c77-eb7b-4a9a-823b-9ede1a846176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:59.240095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-5be52c77-eb7b-4a9a-823b-9ede1a846176 None None] MFA Rules not processed for user `b1c427a987ba48878873734ecf0f86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:00:59.241707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5be52c77-eb7b-4a9a-823b-9ede1a846176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:59.292925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:59.293910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:59.295555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:59.295555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:59.400044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 886/1781] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 14:00:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:00:59.406193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be52c77-eb7b-4a9a-823b-9ede1a846176 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:59.471465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:59.473067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:59.545685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be52c77-eb7b-4a9a-823b-9ede1a846176 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:59.548599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 896/1782] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:59.550590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:59.553852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:59.554391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:59.554883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:59.555918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:59.559371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:59.565288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:59.625419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cd225-a3f2-44d2-88f4-a1c197e0e9cf service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:59.663951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:59.667116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:59.667406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:59.667680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:59.672268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:59.677082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:59.681142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 887/1783] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:59 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:59.682107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:59.688890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:00:59.739091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768573ce-4f9b-48c1-916f-68fc9f44b0ff admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:59.753285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:59.754254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:59.754580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:59.754896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:59.776840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:59.778814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:59.792156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 888/1784] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:00:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:00:59.793559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:59.810011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15927239-1ab0-44f0-aa7b-ffa444d647c0 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:00:59.817111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:00:59.818813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:00:59.819398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:00:59.820018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] PATH_INFO: `/v3/users/205a8a8551604044ac81b836f1c301d7` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:00:59.822138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=205a8a8551604044ac81b836f1c301d7)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:00:59.828721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:00:59.874638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 897/1785] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:00:59 2022] POST /identity/v3/users => generated 283 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:00:59.879324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:00:59.959962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7d027-fded-4105-9b12-d3f665546ce3 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:00.014701 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:00.031262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:00.031262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:00.031262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:00.031262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:00.031262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:00.069624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 898/1786] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:00.071599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:00.108619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653fcb3-f3f5-4e06-94b6-3972ffd9d0a0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:00.121878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:00.124403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:00.125235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:00.127921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:00.128872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:00.131472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:00.190620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 899/1787] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:00.193903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:00.240562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b562a87-e396-412a-815f-3e7bec901a5a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:00.262804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:00.263853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:00.264318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:00.264792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] PATH_INFO: `/v3/projects/5a1319e7f28d4ae2bef14debde65cf95/users/6c685dda73a14aa48bf0987dc2fd04f6/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:00.307907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1319e7f28d4ae2bef14debde65cf95, user_id=6c685dda73a14aa48bf0987dc2fd04f6, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:00.312256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:00.546289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 900/1788] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:00 2022] PUT /identity/v3/projects/5a1319e7f28d4ae2bef14debde65cf95/users/6c685dda73a14aa48bf0987dc2fd04f6/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:00.555043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:00.575142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205a8a8551604044ac81b836f1c301d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:00.586835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205a8a8551604044ac81b836f1c301d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:00.684953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205a8a8551604044ac81b836f1c301d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:00.685889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205a8a8551604044ac81b836f1c301d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:00.720243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205a8a8551604044ac81b836f1c301d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:00.741763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 889/1789] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:00:59 2022] DELETE /identity/v3/users/205a8a8551604044ac81b836f1c301d7 => generated 0 bytes in 948 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:00.742903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:00.748367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:00.788105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de49fd-fa47-47cf-ad72-337c51fdcba8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:00.817209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:00.880426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:00.919599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4df79b-dc35-43ad-b4d6-5e8b3779d551 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:00.939255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f529fd-497c-426d-8e62-64f9a313bf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:00.940201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f529fd-497c-426d-8e62-64f9a313bf23 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:00.940837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f529fd-497c-426d-8e62-64f9a313bf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:00.941300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f529fd-497c-426d-8e62-64f9a313bf23 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:00.942164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f529fd-497c-426d-8e62-64f9a313bf23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:00.943969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f529fd-497c-426d-8e62-64f9a313bf23 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:00.970738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 890/1790] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:00.972625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b788bc-6dd3-429d-a294-3adba57ec514 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:00.979621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b788bc-6dd3-429d-a294-3adba57ec514 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:00.981241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b788bc-6dd3-429d-a294-3adba57ec514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.013073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d1f94-d268-415b-9f72-86297d32634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:01.020723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d1f94-d268-415b-9f72-86297d32634e admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.022296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d1f94-d268-415b-9f72-86297d32634e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.022586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d1f94-d268-415b-9f72-86297d32634e admin admin] PATH_INFO: `/v3/projects/4849da00678148da9762d8de9c28db14/users/aa18ecae17174b88ae770d9bc7b98e00/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.057663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-86b788bc-6dd3-429d-a294-3adba57ec514 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:01.060301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-86b788bc-6dd3-429d-a294-3adba57ec514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:01.071542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 891/1791] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:01.074004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b788bc-6dd3-429d-a294-3adba57ec514 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:01.084899 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d1f94-d268-415b-9f72-86297d32634e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4849da00678148da9762d8de9c28db14, user_id=aa18ecae17174b88ae770d9bc7b98e00, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:01.087969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d1f94-d268-415b-9f72-86297d32634e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:01.089305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b788bc-6dd3-429d-a294-3adba57ec514 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:01.097705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:01.098743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.099307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.099996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] PATH_INFO: `/v3/projects/5a1319e7f28d4ae2bef14debde65cf95/users/6c685dda73a14aa48bf0987dc2fd04f6/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.116268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1319e7f28d4ae2bef14debde65cf95, user_id=6c685dda73a14aa48bf0987dc2fd04f6, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:01.118033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:01.143223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 892/1792] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:01 2022] PUT /identity/v3/projects/5a1319e7f28d4ae2bef14debde65cf95/users/6c685dda73a14aa48bf0987dc2fd04f6/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:01.151877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:01.151877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:01.235984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51561da6-9a12-4544-a356-9b4bb2657177 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:01.254215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 901/1793] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:00 2022] PUT /identity/v3/projects/4849da00678148da9762d8de9c28db14/users/aa18ecae17174b88ae770d9bc7b98e00/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:01.262227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.263990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.267520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.445572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:01.446585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.446952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] MFA Rules not processed for user `6c685dda73a14aa48bf0987dc2fd04f6`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:01.447446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.448937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:01.450154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.572465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 902/1794] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:01.576564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:01.581942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:01.655355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:01.657141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:01.734686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 893/1795] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:01.752619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:01.752619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:01.807749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bfcd8-c2ea-4e26-91cb-76baf1e2b1c2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:01.825786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858251a9-68c8-453f-b583-ef7ef9095783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:01.828850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858251a9-68c8-453f-b583-ef7ef9095783 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.828850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858251a9-68c8-453f-b583-ef7ef9095783 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.828850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858251a9-68c8-453f-b583-ef7ef9095783 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.828850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858251a9-68c8-453f-b583-ef7ef9095783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:01.838708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858251a9-68c8-453f-b583-ef7ef9095783 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:01.875325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 903/1796] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:01.878392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41313-db02-419b-9151-69265e237dda None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.878774 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41313-db02-419b-9151-69265e237dda None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.879058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41313-db02-419b-9151-69265e237dda None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.881245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 904/1797] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:01:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:01:01.882108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41313-db02-419b-9151-69265e237dda None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:01.923896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41313-db02-419b-9151-69265e237dda None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:01.927952 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a8d5f-a88c-4bf7-824f-0ef80818c91d service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:01.933025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcee9f2-daea-478d-9a7c-c28f0e58e6e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:01.933495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcee9f2-daea-478d-9a7c-c28f0e58e6e8 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.933824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcee9f2-daea-478d-9a7c-c28f0e58e6e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.934263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcee9f2-daea-478d-9a7c-c28f0e58e6e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.935221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf6af8-db82-4fee-9306-0779cb0f373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:01.936838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf6af8-db82-4fee-9306-0779cb0f373a admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.937425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf6af8-db82-4fee-9306-0779cb0f373a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.938048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf6af8-db82-4fee-9306-0779cb0f373a admin admin] PATH_INFO: `/v3/projects/4849da00678148da9762d8de9c28db14/users/aa18ecae17174b88ae770d9bc7b98e00/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:01.941860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf6af8-db82-4fee-9306-0779cb0f373a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4849da00678148da9762d8de9c28db14, user_id=aa18ecae17174b88ae770d9bc7b98e00, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:01.957161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcee9f2-daea-478d-9a7c-c28f0e58e6e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:01.959049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcee9f2-daea-478d-9a7c-c28f0e58e6e8 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:01.963527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf6af8-db82-4fee-9306-0779cb0f373a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:01.973741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 894/1798] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:01.990988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:01.991296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:01.992785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:02.002928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 905/1799] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:01 2022] PUT /identity/v3/projects/4849da00678148da9762d8de9c28db14/users/aa18ecae17174b88ae770d9bc7b98e00/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:02.014561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5d145-7874-44a6-b288-31106861e8f8 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:02.015101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5d145-7874-44a6-b288-31106861e8f8 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:02.015497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5d145-7874-44a6-b288-31106861e8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:02.037826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:02.037826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:02.155628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e9f5d145-7874-44a6-b288-31106861e8f8 None None] MFA Rules not processed for user `aa18ecae17174b88ae770d9bc7b98e00`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:02.157096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e9f5d145-7874-44a6-b288-31106861e8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:02.216196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 895/1800] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:01:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:02.235833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:02.238847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:02.278960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac459-c13c-4aea-adb9-2068f130e246 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:02.304801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:02.305736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:02.307848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:02.308192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:02.360687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:02.362425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:02.409857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 906/1801] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:02.426559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 896/1802] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:02.545960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5d145-7874-44a6-b288-31106861e8f8 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:02.753000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5d145-7874-44a6-b288-31106861e8f8 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:02.777221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:02.780146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:02.781034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:02.791991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:02.851692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:02.854739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:02.937261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:02.938955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:02.984017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 907/1803] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:03.028441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebf13e-a591-47ff-9d63-934d300a13fa service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:03.043170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:03.043170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:03.043170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:03.043170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:03.054544 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:03.180137 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:03.182145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:03.316982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:03.320125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:03.413212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 897/1804] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:01:03.487857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:03.560077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fe570-a2cd-4be7-927c-60ae40334b6f service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:03.567173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:03.589118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:03.600242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8c83f-45fe-4098-9b5f-84ea2d0c33c5 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:03.602659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:03.602659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:03.602659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:03.602659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] PATH_INFO: `/v3/projects/ba6c04abb9264f44aa904fcaf9732fd0` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:03.603955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6c04abb9264f44aa904fcaf9732fd0)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:03.607515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:03.619319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac683aa-6166-4a84-81a2-8a108b7a4758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:03.620828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac683aa-6166-4a84-81a2-8a108b7a4758 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:03.621529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac683aa-6166-4a84-81a2-8a108b7a4758 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:03.621996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac683aa-6166-4a84-81a2-8a108b7a4758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:03.624240 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fac683aa-6166-4a84-81a2-8a108b7a4758 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:03.750966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac683aa-6166-4a84-81a2-8a108b7a4758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:03.752497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac683aa-6166-4a84-81a2-8a108b7a4758 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:03.782222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 898/1805] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:03.785695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6c04abb9264f44aa904fcaf9732fd0', 'request_id': 'req-5658449d-ad8d-4260-befe-09ecadf6ebf9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:03.794695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6c04abb9264f44aa904fcaf9732fd0', 'request_id': 'req-5658449d-ad8d-4260-befe-09ecadf6ebf9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:03.801918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6c04abb9264f44aa904fcaf9732fd0', 'request_id': 'req-5658449d-ad8d-4260-befe-09ecadf6ebf9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:03.906251 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] The token cache is being invalidate because project ba6c04abb9264f44aa904fcaf9732fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:03.913885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:03.913885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5658449d-ad8d-4260-befe-09ecadf6ebf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:03.913885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 908/1806] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:03 2022] DELETE /identity/v3/projects/ba6c04abb9264f44aa904fcaf9732fd0 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:03.914508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd9dc0-cf75-4e0a-8038-bf2548aa6fe0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:03.914851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd9dc0-cf75-4e0a-8038-bf2548aa6fe0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:03.915259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd9dc0-cf75-4e0a-8038-bf2548aa6fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:03.942693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-73bd9dc0-cf75-4e0a-8038-bf2548aa6fe0 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:03.945602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-73bd9dc0-cf75-4e0a-8038-bf2548aa6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:04.062232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d70c2-3910-4524-a747-f70823841287 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:04.062739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d70c2-3910-4524-a747-f70823841287 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:04.063198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d70c2-3910-4524-a747-f70823841287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:04.073737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 899/1807] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:04.091924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d53d70c2-3910-4524-a747-f70823841287 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:04.094522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d53d70c2-3910-4524-a747-f70823841287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:04.123716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 909/1808] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:04.134491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd9dc0-cf75-4e0a-8038-bf2548aa6fe0 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:04.152661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd9dc0-cf75-4e0a-8038-bf2548aa6fe0 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:04.203969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:04.204717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:04.205007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:04.205277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:04.205981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:01:04.206280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:04.219257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:04.283273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 900/1809] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:01:04.287486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d70c2-3910-4524-a747-f70823841287 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:04.308875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d70c2-3910-4524-a747-f70823841287 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:04.333870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:04.334517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:04.336798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:04.337071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:04.347926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:04.358411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:04.524050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 910/1810] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:04 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:04.528921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:04.548198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744b3a-f404-458e-835c-9051d614d8da admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:04.653054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:04.654994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:04.655846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:04.656391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:04.659420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:04.668173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:04.800920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 901/1811] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:04 2022] POST /identity/v3/users => generated 288 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:04.805268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:04.819618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458798f9-7d4e-48d5-a945-11abf91ad90c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:04.825525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:04.826285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:04.826529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:04.826878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:04.827640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:04.829295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:04.844823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 911/1812] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:04.847910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:04.862567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42261c6-f349-4cf6-9b47-d4ee281dd532 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:04.877446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc607b8-2cca-440e-b2bd-30542af7f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:04.878816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc607b8-2cca-440e-b2bd-30542af7f347 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:04.878816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc607b8-2cca-440e-b2bd-30542af7f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:04.878816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc607b8-2cca-440e-b2bd-30542af7f347 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:04.879447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc607b8-2cca-440e-b2bd-30542af7f347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:04.881060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc607b8-2cca-440e-b2bd-30542af7f347 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:04.901833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 902/1813] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:04.904093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:04.917519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca3c5-2d42-43de-be18-2cdc3eb18310 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:04.923156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:04.923840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:04.924424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:04.924782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] PATH_INFO: `/v3/projects/243a566c633744c3b1169a33d0764344/users/f2c71c35684446d6b8bbc620506bd9c1/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:04.946560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=243a566c633744c3b1169a33d0764344, user_id=f2c71c35684446d6b8bbc620506bd9c1, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:04.949994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:05.124845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 912/1814] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:04 2022] PUT /identity/v3/projects/243a566c633744c3b1169a33d0764344/users/f2c71c35684446d6b8bbc620506bd9c1/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:05.136252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf964e-a9f3-49c7-8854-9b96ef1fc6de None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:05.136252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf964e-a9f3-49c7-8854-9b96ef1fc6de None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:05.136709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf964e-a9f3-49c7-8854-9b96ef1fc6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:05.186801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-83bf964e-a9f3-49c7-8854-9b96ef1fc6de None None] MFA Rules not processed for user `f2c71c35684446d6b8bbc620506bd9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:05.188877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-83bf964e-a9f3-49c7-8854-9b96ef1fc6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:05.335840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 903/1815] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:05.367111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:05.369211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:05.557011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064a45b-db8d-4e85-84fd-ec5889cbad8a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:05.573122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:05.573877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:05.574216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:05.574495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:05.586918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:05.590481 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:05.622098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 913/1816] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:05.653745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:06.610133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:07.559669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:07.779846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b86b88-4135-4f13-985f-260665842560 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:07.780142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b86b88-4135-4f13-985f-260665842560 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:07.780421 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b86b88-4135-4f13-985f-260665842560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:07.801115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-15b86b88-4135-4f13-985f-260665842560 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:07.802360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-15b86b88-4135-4f13-985f-260665842560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:07.877656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 904/1817] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:07.896756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:07.898398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:07.999964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b825415-53f4-43da-973f-4827dd8d882d service glance] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:08.005315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f7a62-1b36-4001-b3a5-6d390b23ac79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:08.006135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f7a62-1b36-4001-b3a5-6d390b23ac79 service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:08.006773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f7a62-1b36-4001-b3a5-6d390b23ac79 service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:08.007275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f7a62-1b36-4001-b3a5-6d390b23ac79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:08.047622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f7a62-1b36-4001-b3a5-6d390b23ac79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:08.049487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f7a62-1b36-4001-b3a5-6d390b23ac79 service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:08.071099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 914/1818] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:01:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:01:08.402845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:08.519179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:10.176675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b86b88-4135-4f13-985f-260665842560 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:10.179912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-15b86b88-4135-4f13-985f-260665842560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:10.311486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:10.351310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b86b88-4135-4f13-985f-260665842560 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:10.365383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:10.366182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:10.366477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:10.368944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] PATH_INFO: `/v3/users/1949477a74a34380bc106dedb6d5cc2a` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:10.370507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1949477a74a34380bc106dedb6d5cc2a)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:10.377381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:10.639733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1949477a74a34380bc106dedb6d5cc2a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:10.649076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1949477a74a34380bc106dedb6d5cc2a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:10.763946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1949477a74a34380bc106dedb6d5cc2a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:10.764637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1949477a74a34380bc106dedb6d5cc2a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:10.782658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1949477a74a34380bc106dedb6d5cc2a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:10.799108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 905/1819] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:10 2022] DELETE /identity/v3/users/1949477a74a34380bc106dedb6d5cc2a => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:10.803147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34ecff-438f-4170-85b6-e209e1ea52e5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:10.803965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34ecff-438f-4170-85b6-e209e1ea52e5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:10.804506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34ecff-438f-4170-85b6-e209e1ea52e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:10.835291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6e34ecff-438f-4170-85b6-e209e1ea52e5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:10.837712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6e34ecff-438f-4170-85b6-e209e1ea52e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:10.921935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 915/1820] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:10.927789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:10.930318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:11.004588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd172d4b-be87-467f-a7ce-d64fe5a3fbc2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:11.008678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babcfe1-e9ec-4d2d-842c-109c9292ba21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:11.009238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babcfe1-e9ec-4d2d-842c-109c9292ba21 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:11.009570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babcfe1-e9ec-4d2d-842c-109c9292ba21 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:11.009803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babcfe1-e9ec-4d2d-842c-109c9292ba21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:11.021358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babcfe1-e9ec-4d2d-842c-109c9292ba21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:11.022925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babcfe1-e9ec-4d2d-842c-109c9292ba21 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:11.034492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 906/1821] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:11.212051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:11.543581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e19f4-c9e1-47c2-92de-d2827cf44d6f None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:11.544222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e19f4-c9e1-47c2-92de-d2827cf44d6f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:11.544222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e19f4-c9e1-47c2-92de-d2827cf44d6f None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:11.545631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 916/1822] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:01:11.562239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783d5c6-800e-494d-ab40-06a511c3731f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:11.563121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783d5c6-800e-494d-ab40-06a511c3731f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:11.563567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783d5c6-800e-494d-ab40-06a511c3731f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:11.601195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9783d5c6-800e-494d-ab40-06a511c3731f None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:11.602849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9783d5c6-800e-494d-ab40-06a511c3731f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:11.628070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e19f4-c9e1-47c2-92de-d2827cf44d6f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:11.629961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ba6e19f4-c9e1-47c2-92de-d2827cf44d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:11.664548 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e19f4-c9e1-47c2-92de-d2827cf44d6f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:11.667296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:11.676183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:11.676967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:11.677252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:11.678786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] PATH_INFO: `/v3/projects/fb6fce1a3693467c92e6fcb20e053bf3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:11.680552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6fce1a3693467c92e6fcb20e053bf3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:11.691212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:11.762334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 907/1823] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:01:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:11.892040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6fce1a3693467c92e6fcb20e053bf3', 'request_id': 'req-6b47422d-f468-49fa-977e-7ff5397b6554'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:11.914660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6fce1a3693467c92e6fcb20e053bf3', 'request_id': 'req-6b47422d-f468-49fa-977e-7ff5397b6554'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:11.923648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6fce1a3693467c92e6fcb20e053bf3', 'request_id': 'req-6b47422d-f468-49fa-977e-7ff5397b6554'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.119621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] The token cache is being invalidate because project fb6fce1a3693467c92e6fcb20e053bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:12.121799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.123479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.127869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 917/1824] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:11 2022] DELETE /identity/v3/projects/fb6fce1a3693467c92e6fcb20e053bf3 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:12.133827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783d5c6-800e-494d-ab40-06a511c3731f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:12.145390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:12.147021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:12.169960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783d5c6-800e-494d-ab40-06a511c3731f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:12.270486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47422d-f468-49fa-977e-7ff5397b6554 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:12.296425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:12.297135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:12.297485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:12.297852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:12.302676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:12.331970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:12.334572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:12.363488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 918/1825] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:12.396281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:12.397406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:12.397921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:12.398511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] PATH_INFO: `/v3/users/6c685dda73a14aa48bf0987dc2fd04f6` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:12.399919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c685dda73a14aa48bf0987dc2fd04f6)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:12.402779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:12.776548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c685dda73a14aa48bf0987dc2fd04f6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.791741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c685dda73a14aa48bf0987dc2fd04f6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.873417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c685dda73a14aa48bf0987dc2fd04f6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.874845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c685dda73a14aa48bf0987dc2fd04f6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.890517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0dc876c9-71ff-4c39-b5dc-b49f919cfbb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c685dda73a14aa48bf0987dc2fd04f6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:12.903046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 908/1826] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:12 2022] DELETE /identity/v3/users/6c685dda73a14aa48bf0987dc2fd04f6 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:13.411197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:13.659187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:13.661573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:13.705942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc1c9b-039e-42a7-8cb4-6dafd04e6158 service nova] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:13.902574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:13.902574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:13.902574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:13.902574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] PATH_INFO: `/v3/projects/5a1319e7f28d4ae2bef14debde65cf95` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:13.902956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1319e7f28d4ae2bef14debde65cf95)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:13.903568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:14.078589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1319e7f28d4ae2bef14debde65cf95', 'request_id': 'req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:14.086857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1319e7f28d4ae2bef14debde65cf95', 'request_id': 'req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:14.098965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1319e7f28d4ae2bef14debde65cf95', 'request_id': 'req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:14.357840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] The token cache is being invalidate because project 5a1319e7f28d4ae2bef14debde65cf95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:14.358667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:14.359448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9c3d52bf-2b68-4992-b5c7-5c6879c06eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:14.365278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 919/1827] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:13 2022] DELETE /identity/v3/projects/5a1319e7f28d4ae2bef14debde65cf95 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:14.367039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b2945-d518-404c-9576-e2f018f91374 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:14.367039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b2945-d518-404c-9576-e2f018f91374 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:14.367039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b2945-d518-404c-9576-e2f018f91374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:14.401478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7b8b2945-d518-404c-9576-e2f018f91374 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:14.403189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7b8b2945-d518-404c-9576-e2f018f91374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:14.521512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 909/1828] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:14.531278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f2ef2-1869-47d2-b5f6-a9333c9c0d81 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:14.531982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f2ef2-1869-47d2-b5f6-a9333c9c0d81 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:14.531982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f2ef2-1869-47d2-b5f6-a9333c9c0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:14.554988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-913f2ef2-1869-47d2-b5f6-a9333c9c0d81 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:14.556305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-913f2ef2-1869-47d2-b5f6-a9333c9c0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:14.565684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 920/1829] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:14.571242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b2945-d518-404c-9576-e2f018f91374 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:14.583079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b2945-d518-404c-9576-e2f018f91374 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:14.588975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:14.590124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:14.590626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:14.591087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:14.591996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:01:14.592489 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:14.594342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:14.609044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 910/1830] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:01:14.616492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f2ef2-1869-47d2-b5f6-a9333c9c0d81 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:14.644043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f2ef2-1869-47d2-b5f6-a9333c9c0d81 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:14.655302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:14.659127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:14.659127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:14.659357 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:14.660033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:14.662014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:14.796266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 921/1831] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:14 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:14.797076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:14.813975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2425cc-1694-4919-bbf8-97267fb26c0f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:14.922235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:14.922853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:14.946242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:14.946242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:14.946242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:14.946242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:16.123393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:16.124648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 911/1832] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:14 2022] POST /identity/v3/users => generated 294 bytes in 1327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:16.148809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2bc98-ddea-4593-9c9b-a189418bcdb3 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:16.164968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:16.168328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.168328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.168328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.168328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:16.170517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:16.188471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 922/1833] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:16.193956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:16.207801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d2c9a-ffbf-4bb9-b14f-d3e046af5930 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:16.215059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb2558-b64b-40af-af4a-4208f8edf75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:16.215852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2558-b64b-40af-af4a-4208f8edf75f admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.216282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2558-b64b-40af-af4a-4208f8edf75f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.216696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2558-b64b-40af-af4a-4208f8edf75f admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.217547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb2558-b64b-40af-af4a-4208f8edf75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:16.219794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb2558-b64b-40af-af4a-4208f8edf75f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:16.225203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:16.236281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 912/1834] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:16.241032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:16.253090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815b1cd-821d-469c-a87c-d7ac30eddef3 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:16.257908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2493b0-d8b1-4780-b328-452448d08f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:16.258592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2493b0-d8b1-4780-b328-452448d08f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.258880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2493b0-d8b1-4780-b328-452448d08f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.259180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2493b0-d8b1-4780-b328-452448d08f74 admin admin] PATH_INFO: `/v3/projects/71497c1ded9142479b2cdcb361aabdc4/users/3c96f9586cf24c0080d96d63da2b9285/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.284149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2493b0-d8b1-4780-b328-452448d08f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71497c1ded9142479b2cdcb361aabdc4, user_id=3c96f9586cf24c0080d96d63da2b9285, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:16.289348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2493b0-d8b1-4780-b328-452448d08f74 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:16.354793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4583e-50c5-4990-8279-3a8b0ca7bc99 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.355085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4583e-50c5-4990-8279-3a8b0ca7bc99 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.355386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4583e-50c5-4990-8279-3a8b0ca7bc99 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.356820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 913/1835] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:01:16.365993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eda4e-68a8-4558-af0f-492f01498549 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.366276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eda4e-68a8-4558-af0f-492f01498549 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.366539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eda4e-68a8-4558-af0f-492f01498549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.388345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-590eda4e-68a8-4558-af0f-492f01498549 None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:16.389573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-590eda4e-68a8-4558-af0f-492f01498549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:16.451476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 923/1836] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:16 2022] PUT /identity/v3/projects/71497c1ded9142479b2cdcb361aabdc4/users/3c96f9586cf24c0080d96d63da2b9285/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:16.469501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082409f0-a23b-472b-9d73-5ad2df8c7da8 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.470033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082409f0-a23b-472b-9d73-5ad2df8c7da8 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.470743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082409f0-a23b-472b-9d73-5ad2df8c7da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.508832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 914/1837] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:16.535093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-082409f0-a23b-472b-9d73-5ad2df8c7da8 None None] MFA Rules not processed for user `3c96f9586cf24c0080d96d63da2b9285`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:16.536409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-082409f0-a23b-472b-9d73-5ad2df8c7da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:16.634363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 924/1838] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:16.639365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eda4e-68a8-4558-af0f-492f01498549 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:16.650817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eda4e-68a8-4558-af0f-492f01498549 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:16.712662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:16.713278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.713705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.714125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.714789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:16.716792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:16.841314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 915/1839] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:16 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:16.845591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082409f0-a23b-472b-9d73-5ad2df8c7da8 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:16.858079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082409f0-a23b-472b-9d73-5ad2df8c7da8 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:16.894341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:16.898745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:16.930317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:16.930317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.930317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.930317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:16.930651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:16.932797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:16.952717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:16.989103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835635b9-9412-4c4d-908f-fe5779e4cf53 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:16.995504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:16.996372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:16.996664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:16.996935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:17.006081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:17.007682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:17.022506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 916/1840] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:17.083331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 925/1841] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:16 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:17.083866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:17.092517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa46ad-9cb1-48a6-b8f5-bee1ffe9736c service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:17.098441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:17.099396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:17.099779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:17.100196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:17.100949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:17.104712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:17.117805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 917/1842] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:17.119530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:17.131469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357b79b-4851-4c78-bdcf-114c2e3a33a7 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:17.156316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69ec92-5af6-420e-986d-0d5cd6fbd911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:17.157118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69ec92-5af6-420e-986d-0d5cd6fbd911 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:17.157469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69ec92-5af6-420e-986d-0d5cd6fbd911 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:17.158551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69ec92-5af6-420e-986d-0d5cd6fbd911 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:17.158670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69ec92-5af6-420e-986d-0d5cd6fbd911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:17.160743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69ec92-5af6-420e-986d-0d5cd6fbd911 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:17.181604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 926/1843] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:17.184268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:17.198199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7150ea-8cba-4a9e-96b5-333d0835f56f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:17.204009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:17.204865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:17.205818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:17.206208 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] PATH_INFO: `/v3/projects/6cc76603460b4778b96200a881151e17/users/f286c0e6775749f09f4ce914ac2175b7/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:17.236924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc76603460b4778b96200a881151e17, user_id=f286c0e6775749f09f4ce914ac2175b7, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:17.240261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:17.408193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 918/1844] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:17 2022] PUT /identity/v3/projects/6cc76603460b4778b96200a881151e17/users/f286c0e6775749f09f4ce914ac2175b7/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:17.435888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52303ee6-6282-42c9-b006-7229c3cf4c6d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:17.436212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52303ee6-6282-42c9-b006-7229c3cf4c6d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:17.439369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52303ee6-6282-42c9-b006-7229c3cf4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:17.475239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-52303ee6-6282-42c9-b006-7229c3cf4c6d None None] MFA Rules not processed for user `f286c0e6775749f09f4ce914ac2175b7`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:17.476766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-52303ee6-6282-42c9-b006-7229c3cf4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:17.596463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 927/1845] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:17.602301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:17.637742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:17.728614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7472c0-2274-4f5d-a081-01edfe764937 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:17.733437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:17.734085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:17.734359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:17.734627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:17.735268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:17.737277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:17.857876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 919/1846] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:17 2022] POST /identity/v3/projects => generated 387 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:17.862038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52303ee6-6282-42c9-b006-7229c3cf4c6d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:17.873620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52303ee6-6282-42c9-b006-7229c3cf4c6d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:17.988925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:17.989679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:17.990067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:17.990723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:17.991562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:17.993998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:18.136460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 928/1847] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:17 2022] POST /identity/v3/users => generated 300 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:01:18.142345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:18.170928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32153329-260b-48b6-aa3b-41bd213a6ca2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:18.181515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:18.182577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:18.184402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:18.184402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:18.187025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:18.202857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:18.256450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 920/1848] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:18.257333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:18.270612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42791d70-7a18-4fae-87b1-d31316ec8a98 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:18.294920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:18.295606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:18.295930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:18.296240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] PATH_INFO: `/v3/projects/f02e41ce7aef49fba6309c802250b003/users/bde19d37e256495d95fb988462962933/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:18.330203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02e41ce7aef49fba6309c802250b003, user_id=bde19d37e256495d95fb988462962933, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:18.332889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:18.460959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 929/1849] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:18 2022] PUT /identity/v3/projects/f02e41ce7aef49fba6309c802250b003/users/bde19d37e256495d95fb988462962933/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:18.465700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:18.475946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec93d65-4f48-43c3-b42f-5cc2b3650ef7 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:18.545279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c1d70-0578-4921-af9a-8876beb0d1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:18.546663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1d70-0578-4921-af9a-8876beb0d1bc admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:18.546946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1d70-0578-4921-af9a-8876beb0d1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:18.547235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1d70-0578-4921-af9a-8876beb0d1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:18.547900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c1d70-0578-4921-af9a-8876beb0d1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:18.549738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c1d70-0578-4921-af9a-8876beb0d1bc admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:18.561122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 921/1850] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:18.565761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:18.577618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3b4d4-ec11-4d6d-9455-7feb499d68ed admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:18.582705 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:18.583421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:18.583691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:18.583961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] PATH_INFO: `/v3/projects/f02e41ce7aef49fba6309c802250b003/users/bde19d37e256495d95fb988462962933/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:18.586276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02e41ce7aef49fba6309c802250b003, user_id=bde19d37e256495d95fb988462962933, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:18.589071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:18.625442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 930/1851] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:18 2022] PUT /identity/v3/projects/f02e41ce7aef49fba6309c802250b003/users/bde19d37e256495d95fb988462962933/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:18.625684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da615e-3bdb-4378-8e7a-3321c1c7614c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:18.625684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da615e-3bdb-4378-8e7a-3321c1c7614c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:18.625684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da615e-3bdb-4378-8e7a-3321c1c7614c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:18.647138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a9da615e-3bdb-4378-8e7a-3321c1c7614c None None] MFA Rules not processed for user `bde19d37e256495d95fb988462962933`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:18.648425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a9da615e-3bdb-4378-8e7a-3321c1c7614c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:18.724374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 922/1852] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:18.736449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:18.738406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:18.764199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:18.802602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41be1-031d-41be-9c31-dbb5de37865f admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:18.806819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:18.807440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:18.807715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:18.808000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:18.818172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:18.819748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:18.830595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 931/1853] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:19.141230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:20.088882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da615e-3bdb-4378-8e7a-3321c1c7614c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:20.098723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da615e-3bdb-4378-8e7a-3321c1c7614c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:20.102558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:20.103196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:20.103474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:20.103749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:20.159984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:20.161409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:20.170233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 923/1854] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:20.901256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:21.158755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:22.066955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:22.162697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:23.334025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:23.504724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:23.907060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:23.929648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:28.937191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:28.947075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77029f-a1b2-435b-9984-62d384edc2c0 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:28.951283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21146a31-1025-4cb0-a80c-44838e9deeaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:28.952226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21146a31-1025-4cb0-a80c-44838e9deeaa service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:28.952493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21146a31-1025-4cb0-a80c-44838e9deeaa service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:28.952771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21146a31-1025-4cb0-a80c-44838e9deeaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:29.059782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21146a31-1025-4cb0-a80c-44838e9deeaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:29.061421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21146a31-1025-4cb0-a80c-44838e9deeaa service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:29.071966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 932/1855] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:32.466474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:32.468599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:32.591584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3be5af-7a92-42e6-8c88-b546c2739f0d service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:32.615578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:32.618065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:32.618340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:32.621612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] PATH_INFO: `/v3/users/5cf40efb96ce47cf95cfbca7faabce43` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:32.624218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf40efb96ce47cf95cfbca7faabce43)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:32.628432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:32.675185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:32.938650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf40efb96ce47cf95cfbca7faabce43'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:32.964276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf40efb96ce47cf95cfbca7faabce43'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:33.084523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf40efb96ce47cf95cfbca7faabce43'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:33.085404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf40efb96ce47cf95cfbca7faabce43'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:33.108236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf40efb96ce47cf95cfbca7faabce43'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:33.117897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 924/1856] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:32 2022] DELETE /identity/v3/users/5cf40efb96ce47cf95cfbca7faabce43 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:33.122435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:33.123097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:33.123097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:33.145571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:33.149324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:33.242640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 933/1857] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:33.274847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:33.277309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:33.431458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d785cb-ce8a-41dd-a3a7-b49986e1c808 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:33.436566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:33.437413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:33.437964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:33.438406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:33.461583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:33.462943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:33.476036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 925/1858] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:34.608741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:34.610297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:34.638087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693903-1811-4d3d-9cff-e96063ade718 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:34.650031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:34.652911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:34.656475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:34.656475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] PATH_INFO: `/v3/projects/b35b0a25114d4145a076f0ed81c2d0b8` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:34.658345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35b0a25114d4145a076f0ed81c2d0b8)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:34.660485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:34.878707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35b0a25114d4145a076f0ed81c2d0b8', 'request_id': 'req-1a04679d-f63a-4179-abbb-06cedc53438b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:34.894797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35b0a25114d4145a076f0ed81c2d0b8', 'request_id': 'req-1a04679d-f63a-4179-abbb-06cedc53438b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:34.912606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35b0a25114d4145a076f0ed81c2d0b8', 'request_id': 'req-1a04679d-f63a-4179-abbb-06cedc53438b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.069433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] The token cache is being invalidate because project b35b0a25114d4145a076f0ed81c2d0b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:35.069862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.071562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.073828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 934/1859] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:34 2022] DELETE /identity/v3/projects/b35b0a25114d4145a076f0ed81c2d0b8 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:35.079549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:35.101269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c937c87-5134-4d1d-91c3-cfefe871e88b service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:35.209513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:35.211046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:35.211325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:35.211774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] PATH_INFO: `/v3/users/aa18ecae17174b88ae770d9bc7b98e00` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:35.214257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa18ecae17174b88ae770d9bc7b98e00)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:35.218475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:35.651359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:35.668764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa18ecae17174b88ae770d9bc7b98e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.680332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa18ecae17174b88ae770d9bc7b98e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.723410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:35.725133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:35.740539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04679d-f63a-4179-abbb-06cedc53438b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:35.745533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:35.748621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:35.748621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:35.748621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] PATH_INFO: `/v3/users/3c96f9586cf24c0080d96d63da2b9285` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:35.748621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c96f9586cf24c0080d96d63da2b9285)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:35.751005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:35.764313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:35.809978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa18ecae17174b88ae770d9bc7b98e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.811329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa18ecae17174b88ae770d9bc7b98e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.849569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d034ff2e-596f-4b9d-a0ae-347490df932c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa18ecae17174b88ae770d9bc7b98e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:35.873618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 926/1860] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:35 2022] DELETE /identity/v3/users/aa18ecae17174b88ae770d9bc7b98e00 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:36.094600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c96f9586cf24c0080d96d63da2b9285'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:36.114809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c96f9586cf24c0080d96d63da2b9285'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:36.114809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c96f9586cf24c0080d96d63da2b9285'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:36.114809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c96f9586cf24c0080d96d63da2b9285'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:36.229307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c96f9586cf24c0080d96d63da2b9285'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:36.258893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 935/1861] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:35 2022] DELETE /identity/v3/users/3c96f9586cf24c0080d96d63da2b9285 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:36.266018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:36.266908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:36.267356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:36.321170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:36.328567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:36.455755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 927/1862] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:36.478457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:36.483335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:36.600427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d27fe-ceca-4f88-bf7b-1a69dd5c742a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:36.617494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85683b6-cf34-4323-b6ee-b8a397036555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:36.618246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85683b6-cf34-4323-b6ee-b8a397036555 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:36.618493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85683b6-cf34-4323-b6ee-b8a397036555 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:36.618803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85683b6-cf34-4323-b6ee-b8a397036555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:36.634370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:36.637119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:36.638480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85683b6-cf34-4323-b6ee-b8a397036555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:36.640546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85683b6-cf34-4323-b6ee-b8a397036555 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:36.657387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 936/1863] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:36.672805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404875-3f3f-4a68-986c-cacca3cefcb6 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:36.686242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:36.687653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:36.688672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:36.689504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] PATH_INFO: `/v3/projects/4849da00678148da9762d8de9c28db14` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:36.696616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=4849da00678148da9762d8de9c28db14)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:36.697353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:36.904448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4849da00678148da9762d8de9c28db14', 'request_id': 'req-94488cfa-8159-46f2-b284-1b8816fd6eed'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:36.915417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4849da00678148da9762d8de9c28db14', 'request_id': 'req-94488cfa-8159-46f2-b284-1b8816fd6eed'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.031084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4849da00678148da9762d8de9c28db14', 'request_id': 'req-94488cfa-8159-46f2-b284-1b8816fd6eed'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.094420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] The token cache is being invalidate because project 4849da00678148da9762d8de9c28db14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:37.094929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.101983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-94488cfa-8159-46f2-b284-1b8816fd6eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.105383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 928/1864] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:36 2022] DELETE /identity/v3/projects/4849da00678148da9762d8de9c28db14 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:37.113032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3493652-8a12-4098-969b-a1c59b1f251a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:37.113533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3493652-8a12-4098-969b-a1c59b1f251a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:37.114681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3493652-8a12-4098-969b-a1c59b1f251a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:37.164631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d3493652-8a12-4098-969b-a1c59b1f251a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:37.166279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d3493652-8a12-4098-969b-a1c59b1f251a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:37.251368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 937/1865] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:37.269412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bef5-d519-4203-9032-9193302aa769 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:37.272684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bef5-d519-4203-9032-9193302aa769 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:37.272684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bef5-d519-4203-9032-9193302aa769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:37.328510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2910bef5-d519-4203-9032-9193302aa769 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:37.330140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2910bef5-d519-4203-9032-9193302aa769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:37.344521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 929/1866] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:37.350948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3493652-8a12-4098-969b-a1c59b1f251a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:37.369350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3493652-8a12-4098-969b-a1c59b1f251a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:37.379241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:37.380326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:37.381032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:37.381711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:37.389946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:01:37.389946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:37.389946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:37.420027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 938/1867] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:01:37.425506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bef5-d519-4203-9032-9193302aa769 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:37.443713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bef5-d519-4203-9032-9193302aa769 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:37.449977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:37.450617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:37.450894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:37.451164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:37.451816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:37.454936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:37.559199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:37.561254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:37.577559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e33d7-db28-4a50-8fc5-6e7a62c7d667 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:37.615807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:37.616664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:37.617043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:37.617419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] PATH_INFO: `/v3/projects/71497c1ded9142479b2cdcb361aabdc4` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:37.619094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71497c1ded9142479b2cdcb361aabdc4)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:37.622201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:37.635945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:37.662618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 930/1868] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:37 2022] POST /identity/v3/projects => generated 377 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:37.669922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:37.687269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:37.767571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d1b67-9783-4d50-896c-3f12dea8a1d8 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:37.783391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71497c1ded9142479b2cdcb361aabdc4', 'request_id': 'req-e41529f8-20f0-456a-a0bd-92dd83279174'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.795817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71497c1ded9142479b2cdcb361aabdc4', 'request_id': 'req-e41529f8-20f0-456a-a0bd-92dd83279174'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.804094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71497c1ded9142479b2cdcb361aabdc4', 'request_id': 'req-e41529f8-20f0-456a-a0bd-92dd83279174'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.885898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:37.891920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:37.891920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:37.891920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:37.891920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:37.893270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:37.952112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] The token cache is being invalidate because project 71497c1ded9142479b2cdcb361aabdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:37.952408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.953010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:37.954654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 939/1869] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:37 2022] DELETE /identity/v3/projects/71497c1ded9142479b2cdcb361aabdc4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:37.959078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:37.960596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:37.975516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41529f8-20f0-456a-a0bd-92dd83279174 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:37.985834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:37.987154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:37.987644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:37.988174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] PATH_INFO: `/v3/users/f286c0e6775749f09f4ce914ac2175b7` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:37.992492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f286c0e6775749f09f4ce914ac2175b7)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:37.995091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:38.018812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 931/1870] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:37 2022] POST /identity/v3/users => generated 289 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:38.024359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:38.027369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:38.041685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a45b-f0b8-4e9d-bfc3-ce63c730494f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:38.047789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:38.048438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:38.049540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:38.055812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:38.055812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:38.055812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:38.064008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 932/1871] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:38.068425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:38.079827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf0535-2926-44f7-8d36-c8ba29c2e8c5 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:38.087829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:38.089468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:38.090015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:38.090596 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:38.091831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:38.095013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:38.110982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 933/1872] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:38.117317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:38.128462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eee7ac-6aed-495c-8841-619e99edea1e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:38.133524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683f832-5294-473f-a275-8415b6b11a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:38.134254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f832-5294-473f-a275-8415b6b11a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:38.134567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f832-5294-473f-a275-8415b6b11a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:38.134841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f832-5294-473f-a275-8415b6b11a88 admin admin] PATH_INFO: `/v3/projects/307f777723bb4793aae69f9e494d135e/users/efcb5a8c86e7406c9fad18c3126cfa26/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:38.152670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683f832-5294-473f-a275-8415b6b11a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=307f777723bb4793aae69f9e494d135e, user_id=efcb5a8c86e7406c9fad18c3126cfa26, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:38.156429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683f832-5294-473f-a275-8415b6b11a88 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:38.300278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f286c0e6775749f09f4ce914ac2175b7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:38.318864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f286c0e6775749f09f4ce914ac2175b7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:38.327660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f286c0e6775749f09f4ce914ac2175b7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:38.331786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f286c0e6775749f09f4ce914ac2175b7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:38.381375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 934/1873] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:38 2022] PUT /identity/v3/projects/307f777723bb4793aae69f9e494d135e/users/efcb5a8c86e7406c9fad18c3126cfa26/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:38.393548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c611a-dd19-4346-9792-5da9af552248 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:38.393871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c611a-dd19-4346-9792-5da9af552248 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:38.394119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c611a-dd19-4346-9792-5da9af552248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:38.426095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a45c611a-dd19-4346-9792-5da9af552248 None None] MFA Rules not processed for user `efcb5a8c86e7406c9fad18c3126cfa26`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:38.427469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a45c611a-dd19-4346-9792-5da9af552248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:38.437999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f286c0e6775749f09f4ce914ac2175b7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:38.466034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 940/1874] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:37 2022] DELETE /identity/v3/users/f286c0e6775749f09f4ce914ac2175b7 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:38.530451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 935/1875] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:38.535884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:38.539890 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:38.630344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d89ef-048b-4d77-81d0-b5cee2077a23 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:38.635189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:38.635847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:38.636125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:38.636393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:38.637053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:38.639108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:38.800347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 941/1876] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:38 2022] POST /identity/v3/projects => generated 377 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:01:38.806137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c611a-dd19-4346-9792-5da9af552248 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:38.889305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c611a-dd19-4346-9792-5da9af552248 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:38.982087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:38.983152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:38.983707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:38.984049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:38.984813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:38.987071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.035825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:39.038222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:39.056228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:39.064915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e97e4-cdbc-473a-a08e-99f1de989e10 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:39.078521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:39.079271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.079554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.079831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] PATH_INFO: `/v3/projects/6cc76603460b4778b96200a881151e17` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.081097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc76603460b4778b96200a881151e17)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:39.083301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.129839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 936/1877] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:38 2022] POST /identity/v3/users => generated 295 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:39.136150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:39.147761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88f6fc-a340-4a0c-8b72-23de72bd9f08 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:39.153281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:39.153962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.154247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.154520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.155230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:39.157017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.160972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:39.173697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 937/1878] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:39.178505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:39.191891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ea9fa-baf9-4d1a-9006-4864195a0d72 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:39.197569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:39.198379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.198659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.198930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] PATH_INFO: `/v3/projects/9ae1b1dd0cea4b64901600d370077fc2/users/8706526ce3f64e31b9cd4feaa1f8c10a/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.219790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae1b1dd0cea4b64901600d370077fc2, user_id=8706526ce3f64e31b9cd4feaa1f8c10a, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:39.224845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.240540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc76603460b4778b96200a881151e17', 'request_id': 'req-50f7b544-fae8-4285-92cf-6fc6ff540a15'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.244987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc76603460b4778b96200a881151e17', 'request_id': 'req-50f7b544-fae8-4285-92cf-6fc6ff540a15'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.251889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc76603460b4778b96200a881151e17', 'request_id': 'req-50f7b544-fae8-4285-92cf-6fc6ff540a15'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.337562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] The token cache is being invalidate because project 6cc76603460b4778b96200a881151e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:39.338200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.339334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 938/1879] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:39 2022] PUT /identity/v3/projects/9ae1b1dd0cea4b64901600d370077fc2/users/8706526ce3f64e31b9cd4feaa1f8c10a/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:39.339552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.342109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 942/1880] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:39 2022] DELETE /identity/v3/projects/6cc76603460b4778b96200a881151e17 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:39.345661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:39.347596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:39.348500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:39.349719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:39.436109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7b544-fae8-4285-92cf-6fc6ff540a15 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:39.452355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb64ee-0682-4153-8da1-887d805067da admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:39.460725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:39.461845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.462335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.462837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.464551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:39.467779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.488149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 939/1881] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:39.493283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:39.504153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e86efc-e190-4256-b213-061181bc9b86 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:39.509182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cc5ee-a70a-4abd-8d0a-26b70d6f1c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:39.509892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cc5ee-a70a-4abd-8d0a-26b70d6f1c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.510669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cc5ee-a70a-4abd-8d0a-26b70d6f1c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.511028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cc5ee-a70a-4abd-8d0a-26b70d6f1c86 admin admin] PATH_INFO: `/v3/projects/9ae1b1dd0cea4b64901600d370077fc2/users/8706526ce3f64e31b9cd4feaa1f8c10a/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.513147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cc5ee-a70a-4abd-8d0a-26b70d6f1c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae1b1dd0cea4b64901600d370077fc2, user_id=8706526ce3f64e31b9cd4feaa1f8c10a, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:39.515879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cc5ee-a70a-4abd-8d0a-26b70d6f1c86 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.528229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 940/1882] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:39 2022] PUT /identity/v3/projects/9ae1b1dd0cea4b64901600d370077fc2/users/8706526ce3f64e31b9cd4feaa1f8c10a/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:39.530779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:39.531710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.532097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.532458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] PATH_INFO: `/v3/users/bde19d37e256495d95fb988462962933` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.533789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde19d37e256495d95fb988462962933)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:39.536035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.540142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.540432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.540707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.563286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:39.581318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] MFA Rules not processed for user `8706526ce3f64e31b9cd4feaa1f8c10a`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:39.582614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:39.634459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:39.673050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 941/1883] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:39.684873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:39.686778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:39.754094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde19d37e256495d95fb988462962933'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.756805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558aafb-d744-4f9b-bb1a-f504bc33db5f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:39.763438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:39.764222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:39.764613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:39.764980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:39.765521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde19d37e256495d95fb988462962933'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.766495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde19d37e256495d95fb988462962933'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.767281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde19d37e256495d95fb988462962933'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.767563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:39.840660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:39.842226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:39.852991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 942/1884] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:01:39.858827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5e688cd2-a55d-449e-865c-7f848479eb69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde19d37e256495d95fb988462962933'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:39.890133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 943/1885] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:39 2022] DELETE /identity/v3/users/bde19d37e256495d95fb988462962933 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:40.471591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:40.487522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:40.543693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb19b35-6937-41cb-b0e0-b1646008d249 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:40.603121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:40.603783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:40.604557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:40.604817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] PATH_INFO: `/v3/projects/f02e41ce7aef49fba6309c802250b003` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:40.606058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02e41ce7aef49fba6309c802250b003)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:40.608021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:40.824252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02e41ce7aef49fba6309c802250b003', 'request_id': 'req-31d91b44-75b3-49b2-94ef-bca5366d03bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:40.840758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02e41ce7aef49fba6309c802250b003', 'request_id': 'req-31d91b44-75b3-49b2-94ef-bca5366d03bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:41.037517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02e41ce7aef49fba6309c802250b003', 'request_id': 'req-31d91b44-75b3-49b2-94ef-bca5366d03bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:41.050337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] The token cache is being invalidate because project f02e41ce7aef49fba6309c802250b003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:41.050631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:41.051382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-31d91b44-75b3-49b2-94ef-bca5366d03bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:41.052885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 943/1886] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:40 2022] DELETE /identity/v3/projects/f02e41ce7aef49fba6309c802250b003 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:41.059022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0060cab-be31-4791-b777-cd09c8b73846 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:41.059506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0060cab-be31-4791-b777-cd09c8b73846 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:41.059719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0060cab-be31-4791-b777-cd09c8b73846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:41.079925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c0060cab-be31-4791-b777-cd09c8b73846 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:41.081159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c0060cab-be31-4791-b777-cd09c8b73846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:41.148803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 944/1887] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:41.158085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405a0bb-a228-4649-b315-7c841be31baf None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:41.158378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405a0bb-a228-4649-b315-7c841be31baf None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:41.158673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405a0bb-a228-4649-b315-7c841be31baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:41.179126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0405a0bb-a228-4649-b315-7c841be31baf None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:41.180338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0405a0bb-a228-4649-b315-7c841be31baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:41.188453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 944/1888] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:41.192998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0060cab-be31-4791-b777-cd09c8b73846 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:41.204229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0060cab-be31-4791-b777-cd09c8b73846 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:41.209339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:41.210008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:41.210289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:41.210563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:41.211253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:01:41.218797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:41.218797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:41.226934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 945/1889] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:01:41.234254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405a0bb-a228-4649-b315-7c841be31baf None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:41.246011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405a0bb-a228-4649-b315-7c841be31baf None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:41.251577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:41.252202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:41.252476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:41.252744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:41.253385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:41.255329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:41.539138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 945/1890] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:41 2022] POST /identity/v3/projects => generated 379 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:41.540583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:41.549339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676682b-279d-42dd-a983-a5cc44fbb171 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:41.608777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:41.609389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:41.609669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:41.609968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:41.610656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:41.612680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:41.870315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 946/1891] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:41 2022] POST /identity/v3/users => generated 290 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:41.876143 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:41.912097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace89a5c-4737-400a-b971-16865f92181e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:41.919916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:41.920555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:41.920836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:41.921180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:41.921872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:41.923552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:41.937757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 946/1892] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:41.942801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:41.954401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c05ac1-790f-4065-9197-613067c69a76 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:41.959796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca5069-b71a-4990-8e2c-e747d2b09976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:41.960518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5069-b71a-4990-8e2c-e747d2b09976 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:41.960888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5069-b71a-4990-8e2c-e747d2b09976 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:41.961263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5069-b71a-4990-8e2c-e747d2b09976 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:41.962197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca5069-b71a-4990-8e2c-e747d2b09976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:41.965037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca5069-b71a-4990-8e2c-e747d2b09976 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:41.984026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 947/1893] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:41.987976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:42.005697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a53e4-6abf-4aff-84e7-06784fe6db6b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:42.022588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:42.022588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:42.022588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:42.022588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] PATH_INFO: `/v3/projects/836574b842fd4337ac98ff30b98bc3fd/users/04c1a1786f8e41b5af4351373f5a5de9/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:42.022953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:42.032053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836574b842fd4337ac98ff30b98bc3fd, user_id=04c1a1786f8e41b5af4351373f5a5de9, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:42.034899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:42.148400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 947/1894] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:41 2022] PUT /identity/v3/projects/836574b842fd4337ac98ff30b98bc3fd/users/04c1a1786f8e41b5af4351373f5a5de9/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:42.162410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c0824-d099-4a02-a9af-c9a43de9a0f6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:42.162943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c0824-d099-4a02-a9af-c9a43de9a0f6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:42.162943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c0824-d099-4a02-a9af-c9a43de9a0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:42.198820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:42.207112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-0a1c0824-d099-4a02-a9af-c9a43de9a0f6 None None] MFA Rules not processed for user `04c1a1786f8e41b5af4351373f5a5de9`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:42.208845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0a1c0824-d099-4a02-a9af-c9a43de9a0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:42.294829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 948/1895] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:42.299474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:42.368133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fefaee-b76a-44a5-b041-af8a9bdd5ec0 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:42.374072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:42.375689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:42.375689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:42.375689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:42.375962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:42.377751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:42.492638 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 948/1896] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:42 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:42.498249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c0824-d099-4a02-a9af-c9a43de9a0f6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:42.509576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c0824-d099-4a02-a9af-c9a43de9a0f6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:42.593110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:42.593761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:42.594191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:42.594469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:42.595162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:42.597262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:42.705806 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 949/1897] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:42 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:42.709312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:42.728951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727e2e3-95b8-4b90-b5d1-dfba64a2a363 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:42.736670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:42.737813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:42.738319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:42.738762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:42.739908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:42.742933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:42.760836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 949/1898] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:42.763260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:42.774034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d0db3-98a9-4b30-abae-cef453fc0f5d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:42.779425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:42.780116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:42.780414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:42.780659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] PATH_INFO: `/v3/projects/63345b32696f46d7ac2fe13419d49e30/users/08c08ecf4813428a810d79fe8e7ab2e7/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:42.797659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63345b32696f46d7ac2fe13419d49e30, user_id=08c08ecf4813428a810d79fe8e7ab2e7, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:42.800400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:42.919247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 950/1899] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:42 2022] PUT /identity/v3/projects/63345b32696f46d7ac2fe13419d49e30/users/08c08ecf4813428a810d79fe8e7ab2e7/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:42.923984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:42.935887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1b673-27ad-4222-bdfc-ed3880dabc4e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:43.018326 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d2192-bc86-4317-a0d3-98dcd1987c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:43.019425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d2192-bc86-4317-a0d3-98dcd1987c42 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:43.019922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d2192-bc86-4317-a0d3-98dcd1987c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:43.020417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d2192-bc86-4317-a0d3-98dcd1987c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:43.021652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d2192-bc86-4317-a0d3-98dcd1987c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:43.024712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d2192-bc86-4317-a0d3-98dcd1987c42 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:43.046065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 950/1900] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:43.050946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:43.074593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec53de-bfcd-4852-8bd6-a270b9b39f63 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:43.082809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:43.083512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:43.083797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:43.084073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] PATH_INFO: `/v3/projects/63345b32696f46d7ac2fe13419d49e30/users/08c08ecf4813428a810d79fe8e7ab2e7/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:43.086526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=63345b32696f46d7ac2fe13419d49e30, user_id=08c08ecf4813428a810d79fe8e7ab2e7, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:43.089460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:43.104948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 951/1901] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:43 2022] PUT /identity/v3/projects/63345b32696f46d7ac2fe13419d49e30/users/08c08ecf4813428a810d79fe8e7ab2e7/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:43.126823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb846d12-967d-4d0c-be29-33c069a0a3a5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:43.127257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb846d12-967d-4d0c-be29-33c069a0a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:43.127974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb846d12-967d-4d0c-be29-33c069a0a3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:43.161324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-cb846d12-967d-4d0c-be29-33c069a0a3a5 None None] MFA Rules not processed for user `08c08ecf4813428a810d79fe8e7ab2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:43.162630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cb846d12-967d-4d0c-be29-33c069a0a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:43.218199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:43.268654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 951/1902] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:43.300880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:43.302698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:43.366655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33b99-c0fd-4de5-93eb-7e0ed63c6dcd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:43.371341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:43.376987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:43.376987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:43.376987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:43.380030 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:43.382607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:43.384084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:43.395007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 952/1903] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:43.513116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb846d12-967d-4d0c-be29-33c069a0a3a5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:43.526968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb846d12-967d-4d0c-be29-33c069a0a3a5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:43.533937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:43.534532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:43.534812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:43.535085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:43.604279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:43.605741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:43.631374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 952/1904] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:47.868049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:47.871715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:47.889526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc3a0-69bb-4240-b261-d25a6975a3aa service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:47.904077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:47.904077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:47.904077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:47.904077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:47.904423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:47.995443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:47.997177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:48.008404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 953/1905] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:49.391659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:49.490506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425018a-b9de-45a7-b18b-3c496950023f service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:49.497519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:49.498395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:49.498820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:49.499176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] PATH_INFO: `/v3/users/04c1a1786f8e41b5af4351373f5a5de9` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:49.500518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c1a1786f8e41b5af4351373f5a5de9)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:49.503384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:49.741714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c1a1786f8e41b5af4351373f5a5de9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:49.753858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c1a1786f8e41b5af4351373f5a5de9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:49.774651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:49.778457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:49.798744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec050d-4716-4896-aec8-d2dfc1d87dbd service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:49.804207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:49.805026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:49.805415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:49.805842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] PATH_INFO: `/v3/users/caf7ab33a52b4a75addc9f67dc0dc712` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:49.807296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf7ab33a52b4a75addc9f67dc0dc712)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:49.809669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:49.847435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c1a1786f8e41b5af4351373f5a5de9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:49.848251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c1a1786f8e41b5af4351373f5a5de9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:49.866648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-61ef879b-0461-4a24-a1bc-ff19d2a06f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c1a1786f8e41b5af4351373f5a5de9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:49.868919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:49.877291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 953/1906] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:49 2022] DELETE /identity/v3/users/04c1a1786f8e41b5af4351373f5a5de9 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:49.882812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:49.883266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:49.883664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:49.909360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:49.911607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:50.052180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf7ab33a52b4a75addc9f67dc0dc712'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:50.064300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 954/1907] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:50.069811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf7ab33a52b4a75addc9f67dc0dc712'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:50.071567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf7ab33a52b4a75addc9f67dc0dc712'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:50.072744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf7ab33a52b4a75addc9f67dc0dc712'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:50.072902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:50.075588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:50.150420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ea39dbff-8539-4434-b22b-db4993b556a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf7ab33a52b4a75addc9f67dc0dc712'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:50.162596 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2257b7-8d4a-4b44-bbe4-45921c0640fc None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:50.170054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:50.171149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:50.172411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:50.172910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:50.176034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:50.180513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 954/1908] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:49 2022] DELETE /identity/v3/users/caf7ab33a52b4a75addc9f67dc0dc712 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:50.188953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:50.192697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:50.192697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:50.202124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:50.205115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:50.216720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:50.218399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:50.225426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 955/1909] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:50.337550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 955/1910] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:50.348849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:50.351233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:50.368099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98534e7-2f8e-4ddc-8cac-5ab4da48b4aa service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:50.445982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:50.447207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:50.447580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:50.448370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:50.466511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:50.469086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:50.486577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 956/1911] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:50.882197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:50.886433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:50.902694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:50.912409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe98d9-8bb9-4d0e-9ea3-df764d69c969 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:50.920962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:50.922266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:50.922916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:50.923214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] PATH_INFO: `/v3/projects/836574b842fd4337ac98ff30b98bc3fd` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:50.925799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=836574b842fd4337ac98ff30b98bc3fd)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:50.931612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:51.041763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:51.044424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:51.069945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836574b842fd4337ac98ff30b98bc3fd', 'request_id': 'req-a820db32-3c28-4bc6-8470-a67e69a85358'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.076010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303c64b-86b3-4351-9d71-03d0bc62e822 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:51.082767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836574b842fd4337ac98ff30b98bc3fd', 'request_id': 'req-a820db32-3c28-4bc6-8470-a67e69a85358'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.095034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836574b842fd4337ac98ff30b98bc3fd', 'request_id': 'req-a820db32-3c28-4bc6-8470-a67e69a85358'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.165245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:51.183751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:51.184539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:51.184915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:51.185284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] PATH_INFO: `/v3/projects/6f070626fcb447488cc42dbd27b71467` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:51.186747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f070626fcb447488cc42dbd27b71467)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:51.189371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:51.238920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] The token cache is being invalidate because project 836574b842fd4337ac98ff30b98bc3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:51.239289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.241285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.245010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 956/1912] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:50 2022] DELETE /identity/v3/projects/836574b842fd4337ac98ff30b98bc3fd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:51.249862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:51.252344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:51.273041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820db32-3c28-4bc6-8470-a67e69a85358 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:51.285531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:51.286339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:51.286621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:51.286894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] PATH_INFO: `/v3/users/08c08ecf4813428a810d79fe8e7ab2e7` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:51.288312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c08ecf4813428a810d79fe8e7ab2e7)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:51.290370 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:51.333665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f070626fcb447488cc42dbd27b71467', 'request_id': 'req-8076a326-8a31-4788-9b18-dd18a81e9a58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.342061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f070626fcb447488cc42dbd27b71467', 'request_id': 'req-8076a326-8a31-4788-9b18-dd18a81e9a58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.458287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f070626fcb447488cc42dbd27b71467', 'request_id': 'req-8076a326-8a31-4788-9b18-dd18a81e9a58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.470198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] The token cache is being invalidate because project 6f070626fcb447488cc42dbd27b71467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:51.470492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.471169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.472738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 957/1913] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:51 2022] DELETE /identity/v3/projects/6f070626fcb447488cc42dbd27b71467 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:51.476924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:51.478352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:51.553447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076a326-8a31-4788-9b18-dd18a81e9a58 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:51.558875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:51.559550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:51.559831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:51.560105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] PATH_INFO: `/v3/users/a29690e0b73e4551853ee847744428b5` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:51.561257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29690e0b73e4551853ee847744428b5)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:51.563210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:51.594243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c08ecf4813428a810d79fe8e7ab2e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.601400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c08ecf4813428a810d79fe8e7ab2e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.602173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c08ecf4813428a810d79fe8e7ab2e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.602963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c08ecf4813428a810d79fe8e7ab2e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.751543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c08ecf4813428a810d79fe8e7ab2e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.769475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 957/1914] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:51 2022] DELETE /identity/v3/users/08c08ecf4813428a810d79fe8e7ab2e7 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:51.828113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29690e0b73e4551853ee847744428b5'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.837627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29690e0b73e4551853ee847744428b5'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.991582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29690e0b73e4551853ee847744428b5'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:51.992247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29690e0b73e4551853ee847744428b5'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:52.007015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:52.007815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29690e0b73e4551853ee847744428b5'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:52.014484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 958/1915] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:51 2022] DELETE /identity/v3/users/a29690e0b73e4551853ee847744428b5 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:52.415661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:52.419458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:52.613119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:52.619231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36767e-9a27-46eb-9a6c-ad0625f73acd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:52.724379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:52.725061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:52.725358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:52.725628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] PATH_INFO: `/v3/projects/63345b32696f46d7ac2fe13419d49e30` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:52.727060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] RBAC: Authorizing `identity:delete_project(project_id=63345b32696f46d7ac2fe13419d49e30)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:52.729097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:52.800269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:52.804466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:52.835262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3808-c2ca-487a-b122-a9e9153ce2cd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:52.846311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:52.847510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:52.847934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:52.848429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] PATH_INFO: `/v3/projects/37351b55a4f54ae4b5843d3d1271fdf5` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:52.860951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37351b55a4f54ae4b5843d3d1271fdf5)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:52.860951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:52.934956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63345b32696f46d7ac2fe13419d49e30', 'request_id': 'req-b259335d-56a9-4402-9ab1-e6f88c4e546a'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:52.944422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63345b32696f46d7ac2fe13419d49e30', 'request_id': 'req-b259335d-56a9-4402-9ab1-e6f88c4e546a'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:52.951729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63345b32696f46d7ac2fe13419d49e30', 'request_id': 'req-b259335d-56a9-4402-9ab1-e6f88c4e546a'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.031333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37351b55a4f54ae4b5843d3d1271fdf5', 'request_id': 'req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.038510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] The token cache is being invalidate because project 63345b32696f46d7ac2fe13419d49e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:53.038810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.039514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b259335d-56a9-4402-9ab1-e6f88c4e546a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.041943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 958/1916] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:52 2022] DELETE /identity/v3/projects/63345b32696f46d7ac2fe13419d49e30 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:53.045162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37351b55a4f54ae4b5843d3d1271fdf5', 'request_id': 'req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.046699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825d98-95de-4582-8ec7-f9864c39e22c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.047090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825d98-95de-4582-8ec7-f9864c39e22c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.047269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825d98-95de-4582-8ec7-f9864c39e22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.071346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d2825d98-95de-4582-8ec7-f9864c39e22c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:53.073898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d2825d98-95de-4582-8ec7-f9864c39e22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:53.153173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37351b55a4f54ae4b5843d3d1271fdf5', 'request_id': 'req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.164316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 959/1917] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:53.174409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19551fab-15d4-4c10-83dc-267c004dd37f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.174933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19551fab-15d4-4c10-83dc-267c004dd37f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.175565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19551fab-15d4-4c10-83dc-267c004dd37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.175747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] The token cache is being invalidate because project 37351b55a4f54ae4b5843d3d1271fdf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:01:53.176134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.176874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7f28103e-206f-4dc2-94ee-2a0b8030ffa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:01:53.178846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 959/1918] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:01:52 2022] DELETE /identity/v3/projects/37351b55a4f54ae4b5843d3d1271fdf5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:53.191956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248448b-c204-478a-87ab-9fb5a3ba4cd7 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.192498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248448b-c204-478a-87ab-9fb5a3ba4cd7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.193388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248448b-c204-478a-87ab-9fb5a3ba4cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.217272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-19551fab-15d4-4c10-83dc-267c004dd37f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:53.219666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-19551fab-15d4-4c10-83dc-267c004dd37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:53.224113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f248448b-c204-478a-87ab-9fb5a3ba4cd7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:53.226683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f248448b-c204-478a-87ab-9fb5a3ba4cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:53.227763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 960/1919] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:53.234349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19551fab-15d4-4c10-83dc-267c004dd37f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.238416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 960/1920] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:53.249845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19551fab-15d4-4c10-83dc-267c004dd37f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.253390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.254288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.254731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.254907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c866e-c7fd-4536-b366-908ed8a2004d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.255219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.255389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c866e-c7fd-4536-b366-908ed8a2004d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.255720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c866e-c7fd-4536-b366-908ed8a2004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.256205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:01:53.256611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.264327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.284566 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-375c866e-c7fd-4536-b366-908ed8a2004d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:53.286108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-375c866e-c7fd-4536-b366-908ed8a2004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:53.291525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 961/1921] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:01:53.296011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.297449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 961/1922] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:01:53.313829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c866e-c7fd-4536-b366-908ed8a2004d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.331005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96b248-549e-41b3-942c-2bad69b6cb9b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.336823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c866e-c7fd-4536-b366-908ed8a2004d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.345524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.345524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.345524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.345524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.346033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.346033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.346229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.346229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.346229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.346229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.346660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:01:53.346660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.347840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.366455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 962/1923] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:01:53.371940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.383393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fd09d-80d4-43c7-962f-ca9fc3c1905e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.398749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.398749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.398749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.398749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.400780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.400780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.467028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 962/1924] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:01:53.478867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.539557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 963/1925] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:01:53.546077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.560195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:53.630688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff263a6-4a9c-444a-aa6c-6f8accc31b8d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.716021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.716756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.717106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.717512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.718404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.722180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.722459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209734fb-8a30-4893-b2e4-e8d4413c8a7e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.730013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.732108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.732657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.733330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.734845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.737980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.827011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 963/1926] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/users => generated 283 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:53.832581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.838726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 964/1927] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:53 2022] POST /identity/v3/users => generated 284 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:53.842996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.855715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa39753-02e7-4fbd-8343-f9855fdf10a9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.862386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e019-4091-4927-9ed5-1266c6d03a32 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.863546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.864910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.865074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.865519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.866793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.869974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.870716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.871561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.871988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.872406 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.873338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.875718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.888764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 964/1928] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:53.892857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.906176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 965/1929] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:53.906176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.911099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d622f4-eaa2-450f-87eb-0f6928dfa95f admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.919898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237c454-aa3e-4b6e-937d-1f27fc90005d admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.920476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.921538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.922218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.922775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.923865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.926955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.927715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.928764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.929192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.929532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.930602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:53.934049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:53.936355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:53.946359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 965/1930] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:53.955699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 966/1931] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:53.955956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.957063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:53.963099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641516a-e558-4ea5-a409-6f075438977b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.969923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a81aa1-c197-4134-969e-4c29957197da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.970779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a81aa1-c197-4134-969e-4c29957197da admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.971072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec51be-7533-465d-82e4-9081bc045e0b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:53.971347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a81aa1-c197-4134-969e-4c29957197da admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.971618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a81aa1-c197-4134-969e-4c29957197da admin admin] PATH_INFO: `/v3/projects/6e6210157f504c8da028e8a9f0972dd1/users/c22ab687dd7a4ede8ebbf8753aaa1e00/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.977891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6040b53-7ec5-49a0-aeb0-ed643648ca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:53.979111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040b53-7ec5-49a0-aeb0-ed643648ca6e admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:53.979383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040b53-7ec5-49a0-aeb0-ed643648ca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:53.979661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040b53-7ec5-49a0-aeb0-ed643648ca6e admin admin] PATH_INFO: `/v3/projects/94aad8636c4d418fadee28a9ac4fd20c/users/0f57901b510546a39ba5c6cc6bad6dbb/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:53.996388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a81aa1-c197-4134-969e-4c29957197da admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6210157f504c8da028e8a9f0972dd1, user_id=c22ab687dd7a4ede8ebbf8753aaa1e00, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.000280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a81aa1-c197-4134-969e-4c29957197da admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.001219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6040b53-7ec5-49a0-aeb0-ed643648ca6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=94aad8636c4d418fadee28a9ac4fd20c, user_id=0f57901b510546a39ba5c6cc6bad6dbb, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.004194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6040b53-7ec5-49a0-aeb0-ed643648ca6e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.061398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 966/1932] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:53 2022] PUT /identity/v3/projects/6e6210157f504c8da028e8a9f0972dd1/users/c22ab687dd7a4ede8ebbf8753aaa1e00/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:54.072463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea789-7fe9-4931-a0ed-2bb6abcbaa19 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.072722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea789-7fe9-4931-a0ed-2bb6abcbaa19 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.073003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea789-7fe9-4931-a0ed-2bb6abcbaa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.085763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 967/1933] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:53 2022] PUT /identity/v3/projects/94aad8636c4d418fadee28a9ac4fd20c/users/0f57901b510546a39ba5c6cc6bad6dbb/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:54.098758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b11f9-c9f7-490c-b6a4-98bad66a0e85 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.099251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b11f9-c9f7-490c-b6a4-98bad66a0e85 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.099695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b11f9-c9f7-490c-b6a4-98bad66a0e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.114790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e22ea789-7fe9-4931-a0ed-2bb6abcbaa19 None None] MFA Rules not processed for user `c22ab687dd7a4ede8ebbf8753aaa1e00`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:54.120767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e22ea789-7fe9-4931-a0ed-2bb6abcbaa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:54.141865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-5d8b11f9-c9f7-490c-b6a4-98bad66a0e85 None None] MFA Rules not processed for user `0f57901b510546a39ba5c6cc6bad6dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:54.143895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5d8b11f9-c9f7-490c-b6a4-98bad66a0e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:54.269750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 967/1934] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:54.271610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea789-7fe9-4931-a0ed-2bb6abcbaa19 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.278675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 968/1935] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:54.284744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b11f9-c9f7-490c-b6a4-98bad66a0e85 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.356176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b11f9-c9f7-490c-b6a4-98bad66a0e85 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.357576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea789-7fe9-4931-a0ed-2bb6abcbaa19 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.361530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.362194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.362473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.362750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.363407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.363833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.364624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.365019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.365329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.365590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.366472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.368609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.588871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 969/1936] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:54 2022] POST /identity/v3/projects => generated 369 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:01:54.593673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.633979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 968/1937] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:01:54 2022] POST /identity/v3/projects => generated 367 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:01:54.639360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.673569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a3df4-f0b0-4134-a05d-4a699f0f7a38 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.716907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51eb9b-4346-401e-a288-3c4dd4f1507c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.722762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.723387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.723666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.723944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.724609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.726778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.781715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.783181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.783518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.783741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.785000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.789687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.828044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 969/1938] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:54 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:01:54.833085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:54.834588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.846684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001fc18-7155-417d-9d28-c525a9b3c731 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.851746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.852509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.852712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.852986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.853677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.855435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.867505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 970/1939] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:54.872812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.883513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b2bcc-99c5-467b-8295-6c6e5087de30 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.890123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.891015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.891345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.891709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] PATH_INFO: `/v3/projects/49eaed77132a47c4aae0fdb2c0b728e8/users/9ec334d58de9451dac5140b9b4e6930d/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.911027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49eaed77132a47c4aae0fdb2c0b728e8, user_id=9ec334d58de9451dac5140b9b4e6930d, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.916520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.924952 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 970/1940] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:01:54 2022] POST /identity/v3/users => generated 291 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:01:54.926470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.938984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c865-cd9b-4fcb-b79e-3e122615609a admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.945223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.945898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.946335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.946735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:54.948341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:54.950059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:54.963896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 971/1941] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:54.968400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:54.980314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d4b1-c95a-458b-9462-5d9a21aee4b1 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:54.987707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:54.988404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:54.988690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:54.988969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] PATH_INFO: `/v3/projects/35fb1d5eb836489a9795d2635d70104c/users/c87e5f03da444f779dcc8b8695e33968/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.011244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fb1d5eb836489a9795d2635d70104c, user_id=c87e5f03da444f779dcc8b8695e33968, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:55.015722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:55.043154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 971/1942] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:54 2022] PUT /identity/v3/projects/49eaed77132a47c4aae0fdb2c0b728e8/users/9ec334d58de9451dac5140b9b4e6930d/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:55.047384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:55.149022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:55.165860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 972/1943] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:54 2022] PUT /identity/v3/projects/35fb1d5eb836489a9795d2635d70104c/users/c87e5f03da444f779dcc8b8695e33968/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:01:55.168307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:55.182090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f1629-4dc1-4338-9d06-eace4b926e77 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:55.389022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:55.389862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:55.390448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:55.390809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.391560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:55.393405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:55.399697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aacd9-f84f-49e0-9dee-2a6580f767ad admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:55.406661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:55.406964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 972/1944] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:01:55.409582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:55.410720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:55.410834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.412059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:55.414558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:55.414865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:55.426850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 973/1945] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:01:55.430552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4e35a-2386-4a2e-9e8f-9cd0c5262b02 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:55.431515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:55.438509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adeb2e-57da-4768-99e8-9b63a1f938aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:55.439221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adeb2e-57da-4768-99e8-9b63a1f938aa admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:55.439509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adeb2e-57da-4768-99e8-9b63a1f938aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:55.439788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adeb2e-57da-4768-99e8-9b63a1f938aa admin admin] PATH_INFO: `/v3/projects/49eaed77132a47c4aae0fdb2c0b728e8/users/9ec334d58de9451dac5140b9b4e6930d/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.442149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adeb2e-57da-4768-99e8-9b63a1f938aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=49eaed77132a47c4aae0fdb2c0b728e8, user_id=9ec334d58de9451dac5140b9b4e6930d, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:55.445145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adeb2e-57da-4768-99e8-9b63a1f938aa admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:55.449136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bdaf5-9d20-45f5-83bb-e874545f29e5 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:55.455191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d12f5-fcf1-4631-804e-0789e7cd4eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:55.455937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d12f5-fcf1-4631-804e-0789e7cd4eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:55.456219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d12f5-fcf1-4631-804e-0789e7cd4eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:55.456498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d12f5-fcf1-4631-804e-0789e7cd4eeb admin admin] PATH_INFO: `/v3/projects/35fb1d5eb836489a9795d2635d70104c/users/c87e5f03da444f779dcc8b8695e33968/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.457996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 973/1946] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:55 2022] PUT /identity/v3/projects/49eaed77132a47c4aae0fdb2c0b728e8/users/9ec334d58de9451dac5140b9b4e6930d/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:55.458931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d12f5-fcf1-4631-804e-0789e7cd4eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fb1d5eb836489a9795d2635d70104c, user_id=c87e5f03da444f779dcc8b8695e33968, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:55.461591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d12f5-fcf1-4631-804e-0789e7cd4eeb admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:55.469114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:55.469404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:55.469681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.474187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 974/1947] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:01:55 2022] PUT /identity/v3/projects/35fb1d5eb836489a9795d2635d70104c/users/c87e5f03da444f779dcc8b8695e33968/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:01:55.484853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c28e-2d13-4a44-8263-ee680c7a82c8 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:55.485147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c28e-2d13-4a44-8263-ee680c7a82c8 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:55.485458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812c28e-2d13-4a44-8263-ee680c7a82c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.512411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] MFA Rules not processed for user `9ec334d58de9451dac5140b9b4e6930d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:55.513827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:55.521305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b812c28e-2d13-4a44-8263-ee680c7a82c8 None None] MFA Rules not processed for user `c87e5f03da444f779dcc8b8695e33968`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:01:55.522770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b812c28e-2d13-4a44-8263-ee680c7a82c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:55.600623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 974/1948] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:55.610233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 975/1949] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:01:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:01:55.614768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:55.616257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:01:55.676135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c937f-5a53-4b40-8405-1783eb7d6038 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:55.680029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:55.680647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:55.680932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:55.681206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:55.691579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:55.693032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:55.703113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 975/1950] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:55.749220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:55.990520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812c28e-2d13-4a44-8263-ee680c7a82c8 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:56.002583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812c28e-2d13-4a44-8263-ee680c7a82c8 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:56.015372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:56.015868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:56.016142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:56.019959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:56.034813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:56.041185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:56.047452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 976/1951] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:56.100239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:56.156297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:56.171139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec7e9d-bbca-4a7d-8f68-5c30986b373d service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:56.175143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407491a-657d-413c-acdc-0c6948ae3ead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:56.175786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407491a-657d-413c-acdc-0c6948ae3ead service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:56.176064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407491a-657d-413c-acdc-0c6948ae3ead service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:56.176335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407491a-657d-413c-acdc-0c6948ae3ead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:56.261035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407491a-657d-413c-acdc-0c6948ae3ead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:56.262770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407491a-657d-413c-acdc-0c6948ae3ead service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:01:56.272997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 976/1952] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:01:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:01:56.586518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:01:59.877640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:01:59.947814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2437a1-61d0-4f88-a1d9-5b752858b019 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:01:59.952708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:01:59.953399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:01:59.953691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:01:59.953994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] PATH_INFO: `/v3/users/0f57901b510546a39ba5c6cc6bad6dbb` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:01:59.955213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f57901b510546a39ba5c6cc6bad6dbb)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:01:59.957127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:00.177569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f57901b510546a39ba5c6cc6bad6dbb'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.185957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f57901b510546a39ba5c6cc6bad6dbb'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.258024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f57901b510546a39ba5c6cc6bad6dbb'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.258699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f57901b510546a39ba5c6cc6bad6dbb'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.281662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f57901b510546a39ba5c6cc6bad6dbb'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:00.289328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 977/1953] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:01:59 2022] DELETE /identity/v3/users/0f57901b510546a39ba5c6cc6bad6dbb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:00.294008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:00.294561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:00.294561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:00.318050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:00.319492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:00.402937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 977/1954] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:00.419202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:00.421028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:00.465031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:00.510731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c6a32-20f7-49f5-80a6-910656ecbade admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:00.518260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:00.519060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:00.519465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:00.519852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:00.531914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:00.533598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:00.545512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 978/1955] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:01.523335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:01.527150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:01.561304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c9b15-a09f-41ab-9420-c30c426f531e None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:01.561304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:01.561304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:01.561304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:01.561661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] PATH_INFO: `/v3/projects/94aad8636c4d418fadee28a9ac4fd20c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:01.561661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94aad8636c4d418fadee28a9ac4fd20c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:01.561661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:01.717289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94aad8636c4d418fadee28a9ac4fd20c', 'request_id': 'req-860be82a-a4c7-4c30-92da-615789f26187'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:01.724023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94aad8636c4d418fadee28a9ac4fd20c', 'request_id': 'req-860be82a-a4c7-4c30-92da-615789f26187'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:01.732090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94aad8636c4d418fadee28a9ac4fd20c', 'request_id': 'req-860be82a-a4c7-4c30-92da-615789f26187'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:01.735158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:01.877107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] The token cache is being invalidate because project 94aad8636c4d418fadee28a9ac4fd20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:01.877416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:01.878055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:01.879561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 978/1956] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:01 2022] DELETE /identity/v3/projects/94aad8636c4d418fadee28a9ac4fd20c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:01.883712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:01.893712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd88d6-242c-4cb0-911c-d6738a9a9bfc service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:01.960026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:01.960955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:01.961121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:01.962924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] PATH_INFO: `/v3/users/c87e5f03da444f779dcc8b8695e33968` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:01.963686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87e5f03da444f779dcc8b8695e33968)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:01.965835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:02.318441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87e5f03da444f779dcc8b8695e33968'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:02.332976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87e5f03da444f779dcc8b8695e33968'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:02.454632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87e5f03da444f779dcc8b8695e33968'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:02.457754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87e5f03da444f779dcc8b8695e33968'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:02.480286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-320bc3a4-314b-46a4-b79e-ba6ad064d84b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87e5f03da444f779dcc8b8695e33968'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:02.493014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 979/1957] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:01 2022] DELETE /identity/v3/users/c87e5f03da444f779dcc8b8695e33968 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:03.107214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:03.108727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:03.135281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be82a-a4c7-4c30-92da-615789f26187 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:03.214770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:03.215800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:03.216189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:03.216649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] PATH_INFO: `/v3/projects/35fb1d5eb836489a9795d2635d70104c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:03.218058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=35fb1d5eb836489a9795d2635d70104c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:03.220198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:03.444691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35fb1d5eb836489a9795d2635d70104c', 'request_id': 'req-bafe1857-652e-4647-b70f-7b5f637cf7be'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:03.477519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35fb1d5eb836489a9795d2635d70104c', 'request_id': 'req-bafe1857-652e-4647-b70f-7b5f637cf7be'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:03.491752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35fb1d5eb836489a9795d2635d70104c', 'request_id': 'req-bafe1857-652e-4647-b70f-7b5f637cf7be'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:03.603666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] The token cache is being invalidate because project 35fb1d5eb836489a9795d2635d70104c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:03.604066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:03.606047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bafe1857-652e-4647-b70f-7b5f637cf7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:03.608273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 979/1958] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:03 2022] DELETE /identity/v3/projects/35fb1d5eb836489a9795d2635d70104c => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:03.613504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a882ca-c695-4f28-90ca-4cf7455e7e81 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:03.614070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a882ca-c695-4f28-90ca-4cf7455e7e81 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:03.614070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a882ca-c695-4f28-90ca-4cf7455e7e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:03.645744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-82a882ca-c695-4f28-90ca-4cf7455e7e81 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:03.647056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-82a882ca-c695-4f28-90ca-4cf7455e7e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:03.718688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:03.765301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 980/1959] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:03.773900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e5ff-77f9-4439-bd5f-07a34e165416 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:03.773900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e5ff-77f9-4439-bd5f-07a34e165416 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:03.773900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e5ff-77f9-4439-bd5f-07a34e165416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:03.811992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-89a2e5ff-77f9-4439-bd5f-07a34e165416 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:03.813387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-89a2e5ff-77f9-4439-bd5f-07a34e165416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:03.827546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a882ca-c695-4f28-90ca-4cf7455e7e81 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:03.833405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 980/1960] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:03.839666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a882ca-c695-4f28-90ca-4cf7455e7e81 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:03.848488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:03.849493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:03.850080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:03.851024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:03.853703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:03.854970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:03.859868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:03.884465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 981/1961] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:02:03.885939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e5ff-77f9-4439-bd5f-07a34e165416 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:03.899168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e5ff-77f9-4439-bd5f-07a34e165416 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:03.906214 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:03.906870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:03.907154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:03.907439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:03.908090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:03.910105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:04.068903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 981/1962] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:03 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:04.074845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:04.097447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a049b2f-9275-474e-a7d2-1c3b5097e296 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:04.233614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:04.234324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:04.234703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:04.235267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:04.236018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:04.238186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:04.415365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 982/1963] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:04 2022] POST /identity/v3/users => generated 293 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:04.419426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:04.434164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843693c-210b-460c-9084-fb4b4f4d6151 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:04.450651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:04.451282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:04.451552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:04.451870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:04.452527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:04.454252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:04.475486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 982/1964] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:04.479792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:04.490736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d15b8-aa4e-4bed-9591-e587313934aa admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:04.497472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79867d16-65ee-4be0-ad49-864d2f0b8315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:04.498138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79867d16-65ee-4be0-ad49-864d2f0b8315 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:04.498422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79867d16-65ee-4be0-ad49-864d2f0b8315 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:04.498700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79867d16-65ee-4be0-ad49-864d2f0b8315 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:04.499363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79867d16-65ee-4be0-ad49-864d2f0b8315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:04.500981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79867d16-65ee-4be0-ad49-864d2f0b8315 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:04.514435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 983/1965] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:04.519715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:04.530585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1e95f-9fa5-49fc-8d53-5762aabd48b1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:04.536001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:04.536642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:04.536929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:04.537210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] PATH_INFO: `/v3/projects/edf7d48d206b4edca544de819e38541e/users/aa53bc0450ee40d9845b24c4087297c9/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:04.554425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf7d48d206b4edca544de819e38541e, user_id=aa53bc0450ee40d9845b24c4087297c9, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:04.558306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:04.686448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 983/1966] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:04 2022] PUT /identity/v3/projects/edf7d48d206b4edca544de819e38541e/users/aa53bc0450ee40d9845b24c4087297c9/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:04.699548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b09d75-31dd-43dc-a4a2-f33350aa0a55 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:04.700080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b09d75-31dd-43dc-a4a2-f33350aa0a55 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:04.700158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b09d75-31dd-43dc-a4a2-f33350aa0a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:04.740710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f5b09d75-31dd-43dc-a4a2-f33350aa0a55 None None] MFA Rules not processed for user `aa53bc0450ee40d9845b24c4087297c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:04.741878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f5b09d75-31dd-43dc-a4a2-f33350aa0a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:04.832598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 984/1967] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:04.836816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:04.871346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:04.960324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afad977-4eab-4c83-96b0-bb28a85c34ed admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:04.967568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:04.969979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:04.970696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:04.971150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:04.971899 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:04.974561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.084887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 984/1968] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:04 2022] POST /identity/v3/projects => generated 385 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:05.091832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b09d75-31dd-43dc-a4a2-f33350aa0a55 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.105545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b09d75-31dd-43dc-a4a2-f33350aa0a55 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:05.177854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:05.218240 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:05.219095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.219724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.220228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.221211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:05.224654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.346359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 985/1969] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:05 2022] POST /identity/v3/users => generated 299 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:05.351522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.364645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65891ba-1671-4ed5-bc20-13c806a8f35a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:05.371432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:05.372080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.372432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.372782 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.373724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:05.375637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.388626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 985/1970] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:05.395470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.418059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c113433-d67a-4a1e-992a-a88729316985 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:05.426707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:05.428083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.428699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.429361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] PATH_INFO: `/v3/projects/f92929a1f8d544419f65f0d3c2345fb4/users/65ec25d4ec9c40878e356c5481bd60bd/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.467395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92929a1f8d544419f65f0d3c2345fb4, user_id=65ec25d4ec9c40878e356c5481bd60bd, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:05.470926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.558570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 986/1971] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:05 2022] PUT /identity/v3/projects/f92929a1f8d544419f65f0d3c2345fb4/users/65ec25d4ec9c40878e356c5481bd60bd/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:05.563382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.573999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0d32c-7073-4e9e-9ec4-5842bc6099d0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:05.640319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0905b6-1891-4b82-9683-a50d3029a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:05.640966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0905b6-1891-4b82-9683-a50d3029a2fa admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.641250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0905b6-1891-4b82-9683-a50d3029a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.641521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0905b6-1891-4b82-9683-a50d3029a2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.642213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0905b6-1891-4b82-9683-a50d3029a2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:05.643814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0905b6-1891-4b82-9683-a50d3029a2fa admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.655398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 986/1972] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:05.659795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.671888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e6502-9373-4daa-80af-ef1bf52a26f2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:05.677965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d833b1e-5e41-4437-bc33-aa8ebca45a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:05.679136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d833b1e-5e41-4437-bc33-aa8ebca45a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.679136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d833b1e-5e41-4437-bc33-aa8ebca45a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.680300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d833b1e-5e41-4437-bc33-aa8ebca45a3a admin admin] PATH_INFO: `/v3/projects/f92929a1f8d544419f65f0d3c2345fb4/users/65ec25d4ec9c40878e356c5481bd60bd/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.681643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d833b1e-5e41-4437-bc33-aa8ebca45a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92929a1f8d544419f65f0d3c2345fb4, user_id=65ec25d4ec9c40878e356c5481bd60bd, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:05.685037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d833b1e-5e41-4437-bc33-aa8ebca45a3a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.698236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 987/1973] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:05 2022] PUT /identity/v3/projects/f92929a1f8d544419f65f0d3c2345fb4/users/65ec25d4ec9c40878e356c5481bd60bd/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:05.717471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.718067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.718447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.745244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1740c48-1c94-477b-b63e-f3c766fd24f3 None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.746475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1740c48-1c94-477b-b63e-f3c766fd24f3 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.746756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1740c48-1c94-477b-b63e-f3c766fd24f3 None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.748591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 988/1974] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:02:05.753791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] MFA Rules not processed for user `65ec25d4ec9c40878e356c5481bd60bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:05.754750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.755037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.755141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:05.755315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.780517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:05.781650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:05.848046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 987/1975] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:05.859227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 989/1976] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:02:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:05.860620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.862328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:05.868716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.870328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:05.884889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e92fa-77a0-47de-9e4d-33de64342018 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:05.890409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:05.896159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:05.896855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.897275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.897716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.914968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:05.918402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.933154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 990/1977] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:05.935699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:05.937086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:05.945510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118bfaf-d1bb-40fb-adb7-dc6fbffe208f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:05.951284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:05.956075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:05.956075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:05.956075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:05.963686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:05.965167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:05.976273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 988/1978] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:06.034062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12375184-b74d-4383-87b8-bda9909fb741 service placement] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:06.039088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:06.039754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:06.040379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:06.040509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] PATH_INFO: `/v3/users/a70246b8e8b842129c99827859b90478` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:06.042049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70246b8e8b842129c99827859b90478)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:06.044350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:06.055515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:06.070030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:06.073097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503a82a-6d69-4d7b-999e-b63b1d189af2 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:06.078809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:06.079479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:06.079765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:06.080040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] PATH_INFO: `/v3/users/aa53bc0450ee40d9845b24c4087297c9` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:06.083076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa53bc0450ee40d9845b24c4087297c9)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:06.083328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:06.251754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70246b8e8b842129c99827859b90478'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.260724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70246b8e8b842129c99827859b90478'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.325009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa53bc0450ee40d9845b24c4087297c9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.337361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa53bc0450ee40d9845b24c4087297c9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.338413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa53bc0450ee40d9845b24c4087297c9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.339178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa53bc0450ee40d9845b24c4087297c9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.408414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70246b8e8b842129c99827859b90478'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.409059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70246b8e8b842129c99827859b90478'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.423957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-470c1d5a-3f4b-4a8b-8732-93da7d71718e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70246b8e8b842129c99827859b90478'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.431613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 991/1979] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:05 2022] DELETE /identity/v3/users/a70246b8e8b842129c99827859b90478 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:06.481058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa53bc0450ee40d9845b24c4087297c9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.507888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 989/1980] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:06 2022] DELETE /identity/v3/users/aa53bc0450ee40d9845b24c4087297c9 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:06.515065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:06.515562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:06.516046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:06.554119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:06.554119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:06.639669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 992/1981] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:06.650229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:06.651776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:06.714776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e74944-7d20-4eca-9e51-73e6e49e47ad admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:06.718592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbc521-c869-4054-a6e2-d7fcd7813450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:06.719207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc521-c869-4054-a6e2-d7fcd7813450 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:06.719511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc521-c869-4054-a6e2-d7fcd7813450 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:06.719789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc521-c869-4054-a6e2-d7fcd7813450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:06.740647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bbc521-c869-4054-a6e2-d7fcd7813450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:06.740831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:06.742169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bbc521-c869-4054-a6e2-d7fcd7813450 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:06.748785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:06.754748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 990/1982] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:06.763732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38df59-84fc-406b-9960-3ae9d6996b5e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:06.777360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:06.780871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:06.780871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:06.780871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] PATH_INFO: `/v3/projects/1c53449be8e448b09a184ea79278bb2a` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:06.782011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c53449be8e448b09a184ea79278bb2a)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:06.784613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:06.949289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c53449be8e448b09a184ea79278bb2a', 'request_id': 'req-a777c993-5510-47bd-bdfd-ecfb145bf441'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:06.955274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c53449be8e448b09a184ea79278bb2a', 'request_id': 'req-a777c993-5510-47bd-bdfd-ecfb145bf441'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.071334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c53449be8e448b09a184ea79278bb2a', 'request_id': 'req-a777c993-5510-47bd-bdfd-ecfb145bf441'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.095661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] The token cache is being invalidate because project 1c53449be8e448b09a184ea79278bb2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:07.096039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.113179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a777c993-5510-47bd-bdfd-ecfb145bf441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.121258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 993/1983] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:06 2022] DELETE /identity/v3/projects/1c53449be8e448b09a184ea79278bb2a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:07.124319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f11b22-9e23-4f29-ad0d-165c0bd9a4d2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.124711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f11b22-9e23-4f29-ad0d-165c0bd9a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.124994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f11b22-9e23-4f29-ad0d-165c0bd9a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.150919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f9f11b22-9e23-4f29-ad0d-165c0bd9a4d2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:07.152255 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f9f11b22-9e23-4f29-ad0d-165c0bd9a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:07.187192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:07.221644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 991/1984] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:07.231773 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf2c1-b68b-4b27-9ff9-b668fbbdb36b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.232235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf2c1-b68b-4b27-9ff9-b668fbbdb36b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.232622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf2c1-b68b-4b27-9ff9-b668fbbdb36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.257442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-afbcf2c1-b68b-4b27-9ff9-b668fbbdb36b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:07.259442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-afbcf2c1-b68b-4b27-9ff9-b668fbbdb36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:07.268108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 994/1985] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:07.272726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f11b22-9e23-4f29-ad0d-165c0bd9a4d2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.287210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f11b22-9e23-4f29-ad0d-165c0bd9a4d2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.292699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.293319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.293579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.293883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.297575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:07.298010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.299845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:07.315145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 992/1986] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:02:07.319021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcf2c1-b68b-4b27-9ff9-b668fbbdb36b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.330575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcf2c1-b68b-4b27-9ff9-b668fbbdb36b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.335599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.336341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.339592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.340298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.341046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.343218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:07.353157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:07.393316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.395116 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:07.410947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee824107-a6e7-4d25-a266-23aa91a8ac8c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.417745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.418475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.418781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.419052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] PATH_INFO: `/v3/projects/edf7d48d206b4edca544de819e38541e` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.420438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf7d48d206b4edca544de819e38541e)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.426362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:07.540727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 995/1987] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:07 2022] POST /identity/v3/projects => generated 393 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:07.541924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.574986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf7d48d206b4edca544de819e38541e', 'request_id': 'req-cb54c6db-575b-42c7-a345-0960e1cb5649'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.582172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf7d48d206b4edca544de819e38541e', 'request_id': 'req-cb54c6db-575b-42c7-a345-0960e1cb5649'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.588611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf7d48d206b4edca544de819e38541e', 'request_id': 'req-cb54c6db-575b-42c7-a345-0960e1cb5649'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.612948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46883c-ea32-4749-9cb3-bdd55128d125 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.670586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.671106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.671382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.671648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.672304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.674391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:07.693330 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] The token cache is being invalidate because project edf7d48d206b4edca544de819e38541e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:07.697504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.697504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:07.697504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 993/1988] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:07 2022] DELETE /identity/v3/projects/edf7d48d206b4edca544de819e38541e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:07.700425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.701912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:07.716289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54c6db-575b-42c7-a345-0960e1cb5649 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.724405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.726042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.726042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.726042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] PATH_INFO: `/v3/users/65ec25d4ec9c40878e356c5481bd60bd` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.726877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ec25d4ec9c40878e356c5481bd60bd)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.728841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:07.801264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 996/1989] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:07 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:07.805440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.807698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:07.820214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966166e-10b4-4fd1-8e9e-94cc89a0d92e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.826018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.826765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.827046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.827338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.827990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.829664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:07.840364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 997/1990] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:02:07.844660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.859563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f352-f751-4c3d-bb17-ac7ea08fa8e6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.865176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.866120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.866415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.866736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.867505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.870381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:07.882249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 998/1991] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:02:07.887176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:07.897666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70e26f-1019-40ee-89c8-ae5a29214f54 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:07.903205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f705463-980a-4139-bd57-c946aa54d8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:07.904008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705463-980a-4139-bd57-c946aa54d8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:07.904329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705463-980a-4139-bd57-c946aa54d8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:07.904624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705463-980a-4139-bd57-c946aa54d8c7 admin admin] PATH_INFO: `/v3/projects/dce10589b8b240e6829f2e158415a1b8/users/65c88d8b86544ab5825adbadb26b1e2e/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:07.921458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f705463-980a-4139-bd57-c946aa54d8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce10589b8b240e6829f2e158415a1b8, user_id=65c88d8b86544ab5825adbadb26b1e2e, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:07.924126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f705463-980a-4139-bd57-c946aa54d8c7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:08.070452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ec25d4ec9c40878e356c5481bd60bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.078005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ec25d4ec9c40878e356c5481bd60bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.079987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ec25d4ec9c40878e356c5481bd60bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.080879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ec25d4ec9c40878e356c5481bd60bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.172161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ec25d4ec9c40878e356c5481bd60bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.178966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 999/1992] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:07 2022] PUT /identity/v3/projects/dce10589b8b240e6829f2e158415a1b8/users/65c88d8b86544ab5825adbadb26b1e2e/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:08.186361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:08.186827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:08.187296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:08.203924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 994/1993] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:07 2022] DELETE /identity/v3/users/65ec25d4ec9c40878e356c5481bd60bd => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:08.240381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:08.242002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:08.303570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:08.333095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1000/1994] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:08.345384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:08.346891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:08.363974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:08.365607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:08.369234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:08.379668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a83b2-7835-4285-9d52-d5e4c556c011 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:08.385376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:08.386057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:08.386321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:08.386595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] PATH_INFO: `/v3/users/c22ab687dd7a4ede8ebbf8753aaa1e00` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:08.387756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c22ab687dd7a4ede8ebbf8753aaa1e00)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:08.389709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:08.411994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008576c-9a29-4046-889c-ebc45edb5664 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:08.416073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae38e1-3175-48fd-a065-bb8fbdce01c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:08.416705 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae38e1-3175-48fd-a065-bb8fbdce01c7 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:08.416984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae38e1-3175-48fd-a065-bb8fbdce01c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:08.417256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae38e1-3175-48fd-a065-bb8fbdce01c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:08.431235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae38e1-3175-48fd-a065-bb8fbdce01c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:08.431235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae38e1-3175-48fd-a065-bb8fbdce01c7 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:08.441301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 995/1995] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:08.635876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c22ab687dd7a4ede8ebbf8753aaa1e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.644187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c22ab687dd7a4ede8ebbf8753aaa1e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.773403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c22ab687dd7a4ede8ebbf8753aaa1e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.774069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c22ab687dd7a4ede8ebbf8753aaa1e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.790594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c22ab687dd7a4ede8ebbf8753aaa1e00'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:08.794707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1001/1996] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:08 2022] DELETE /identity/v3/users/c22ab687dd7a4ede8ebbf8753aaa1e00 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:08.803129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:08.803645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:08.803733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:08.828394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:08.834195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:08.834325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:08.835183 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:08.967689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 996/1997] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:08.972637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdae2b4-7716-45d8-8519-bd4bd0495445 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:08.978804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:08.979808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:08.980093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:08.980607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] PATH_INFO: `/v3/projects/f92929a1f8d544419f65f0d3c2345fb4` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:08.981978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92929a1f8d544419f65f0d3c2345fb4)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:08.983276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:08.984267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:08.985796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:09.081177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460581e-72fc-4bd4-9c69-b71b102b58ac None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:09.085257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274261-0179-4d15-8ba4-db948edc1ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:09.085935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274261-0179-4d15-8ba4-db948edc1ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:09.086213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274261-0179-4d15-8ba4-db948edc1ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:09.086489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274261-0179-4d15-8ba4-db948edc1ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:09.102959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274261-0179-4d15-8ba4-db948edc1ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:09.102959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274261-0179-4d15-8ba4-db948edc1ce1 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:09.109074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 997/1998] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:09.191712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92929a1f8d544419f65f0d3c2345fb4', 'request_id': 'req-8859d572-cb0f-4a39-9b1c-52890e0d5120'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:09.205891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:09.210644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92929a1f8d544419f65f0d3c2345fb4', 'request_id': 'req-8859d572-cb0f-4a39-9b1c-52890e0d5120'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:09.366093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92929a1f8d544419f65f0d3c2345fb4', 'request_id': 'req-8859d572-cb0f-4a39-9b1c-52890e0d5120'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:09.446798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] The token cache is being invalidate because project f92929a1f8d544419f65f0d3c2345fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:09.449009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:09.452948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8859d572-cb0f-4a39-9b1c-52890e0d5120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:09.457281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1002/1999] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:08 2022] DELETE /identity/v3/projects/f92929a1f8d544419f65f0d3c2345fb4 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:09.465129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5899f-58e2-4640-a308-5882ae6df349 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:09.465611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5899f-58e2-4640-a308-5882ae6df349 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:09.466117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5899f-58e2-4640-a308-5882ae6df349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:09.499636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d6c5899f-58e2-4640-a308-5882ae6df349 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:09.501569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d6c5899f-58e2-4640-a308-5882ae6df349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:09.597574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 998/2000] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:09.608502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:09.609062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:09.609427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:09.632752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:09.633821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:09.642144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1003/2001] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:09.646490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5899f-58e2-4640-a308-5882ae6df349 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:09.659401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:09.660133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5899f-58e2-4640-a308-5882ae6df349 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:09.661739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:09.666285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:09.666918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:09.667195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:09.667195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:09.668125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:09.668428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:09.670212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:09.685833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 999/2002] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:02:09.689085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:09.690185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeefb9-5bfa-42a2-b8b8-e0b50abf278c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:09.702238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07763c97-1d94-483d-ba14-404a8674f696 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:09.703499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:09.704067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:09.704348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:09.704646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] PATH_INFO: `/v3/projects/6e6210157f504c8da028e8a9f0972dd1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:09.713951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:09.714165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6210157f504c8da028e8a9f0972dd1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:09.714257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:09.714257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:09.714257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:09.714558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:09.714633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:09.714633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:09.778313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:09.850090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1000/2003] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:09 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:09.855491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:09.906049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6210157f504c8da028e8a9f0972dd1', 'request_id': 'req-0139ccbb-8c3d-4d52-8359-59e7f193d30a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:09.912586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6210157f504c8da028e8a9f0972dd1', 'request_id': 'req-0139ccbb-8c3d-4d52-8359-59e7f193d30a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:09.925510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2065f3-d66e-4856-aacb-fe94dfcd24e3 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:09.987953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:09.988531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:09.988847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:09.991520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:09.992490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:09.994921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:10.020526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6210157f504c8da028e8a9f0972dd1', 'request_id': 'req-0139ccbb-8c3d-4d52-8359-59e7f193d30a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.035469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] The token cache is being invalidate because project 6e6210157f504c8da028e8a9f0972dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:10.035762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.037678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.041271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1004/2004] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:09 2022] DELETE /identity/v3/projects/6e6210157f504c8da028e8a9f0972dd1 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:10.045692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:10.048047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:10.062340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139ccbb-8c3d-4d52-8359-59e7f193d30a admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:10.068399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:10.069593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:10.070227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:10.070824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] PATH_INFO: `/v3/users/9ec334d58de9451dac5140b9b4e6930d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:10.073066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec334d58de9451dac5140b9b4e6930d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:10.077112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:10.136213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1001/2005] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:09 2022] POST /identity/v3/users => generated 287 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:10.140898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:10.142728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:10.156348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c378d-ffd4-49e7-b640-83c54572ba51 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:10.161534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:10.162552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:10.162893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:10.163225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:10.163927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:10.166451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:10.178235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1002/2006] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:02:10.182411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:10.192791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f251c7-82df-44db-a883-b1f4993bb3cb admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:10.197875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:10.198525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:10.198804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:10.199292 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:10.199947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:10.202145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:10.221245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1003/2007] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:02:10.225528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:10.237692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad27fd-d54d-4cb6-91c4-150b7bd904dc admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:10.244331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d164f-c0fe-4823-aeff-006ecdff5060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:10.245044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d164f-c0fe-4823-aeff-006ecdff5060 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:10.245324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d164f-c0fe-4823-aeff-006ecdff5060 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:10.245600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d164f-c0fe-4823-aeff-006ecdff5060 admin admin] PATH_INFO: `/v3/projects/d7387ec4a8da43d39561bb4d6f5d11ad/users/55c90cee381347d9a2c4797a8ae8d580/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:10.272519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d164f-c0fe-4823-aeff-006ecdff5060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7387ec4a8da43d39561bb4d6f5d11ad, user_id=55c90cee381347d9a2c4797a8ae8d580, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:10.278901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d164f-c0fe-4823-aeff-006ecdff5060 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:10.385997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec334d58de9451dac5140b9b4e6930d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.401740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec334d58de9451dac5140b9b4e6930d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.413408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1004/2008] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:10 2022] PUT /identity/v3/projects/d7387ec4a8da43d39561bb4d6f5d11ad/users/55c90cee381347d9a2c4797a8ae8d580/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:10.418610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:10.418894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:10.419167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:10.440870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:10.442122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:10.512665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:10.528028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1005/2009] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:10.538051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:10.539502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:10.541250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec334d58de9451dac5140b9b4e6930d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.542464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec334d58de9451dac5140b9b4e6930d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.586288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec334d58de9451dac5140b9b4e6930d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:10.598164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1005/2010] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:10 2022] DELETE /identity/v3/users/9ec334d58de9451dac5140b9b4e6930d => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:10.663737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f418c7-0122-4e9c-9207-e6ce9cbbdf18 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:10.736255 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f901688-7dac-49c0-9d1e-730cab1d9c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:10.736985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f901688-7dac-49c0-9d1e-730cab1d9c8b service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:10.737320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f901688-7dac-49c0-9d1e-730cab1d9c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:10.737652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f901688-7dac-49c0-9d1e-730cab1d9c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:10.741799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4f901688-7dac-49c0-9d1e-730cab1d9c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:10.812938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f901688-7dac-49c0-9d1e-730cab1d9c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:10.814662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f901688-7dac-49c0-9d1e-730cab1d9c8b service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:10.826738 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1006/2011] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:11.267958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:11.271256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:11.296123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02032ba-fe6f-438a-892f-72e33f730f3d admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:11.303924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:11.304818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:11.305373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:11.306419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] PATH_INFO: `/v3/projects/49eaed77132a47c4aae0fdb2c0b728e8` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:11.307778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49eaed77132a47c4aae0fdb2c0b728e8)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:11.312391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:11.456308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=20,keystone:SELECT=77,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:11.487254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49eaed77132a47c4aae0fdb2c0b728e8', 'request_id': 'req-db89b73f-95bd-423a-b8b1-e758a1f3e120'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:11.497264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49eaed77132a47c4aae0fdb2c0b728e8', 'request_id': 'req-db89b73f-95bd-423a-b8b1-e758a1f3e120'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:11.619816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49eaed77132a47c4aae0fdb2c0b728e8', 'request_id': 'req-db89b73f-95bd-423a-b8b1-e758a1f3e120'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:11.635220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] The token cache is being invalidate because project 49eaed77132a47c4aae0fdb2c0b728e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:11.635526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:11.636689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-db89b73f-95bd-423a-b8b1-e758a1f3e120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:11.640995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1006/2012] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:11 2022] DELETE /identity/v3/projects/49eaed77132a47c4aae0fdb2c0b728e8 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:11.645427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3cd61-8c9f-4088-a949-e69a17c854ad None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:11.646240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3cd61-8c9f-4088-a949-e69a17c854ad None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:11.646358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3cd61-8c9f-4088-a949-e69a17c854ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:11.676861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2ce3cd61-8c9f-4088-a949-e69a17c854ad None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:11.679020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2ce3cd61-8c9f-4088-a949-e69a17c854ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:11.846308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1007/2013] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:11.849599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d51d31-c68d-42ee-ab8e-18ac4bd6e425 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:11.850003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d51d31-c68d-42ee-ab8e-18ac4bd6e425 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:11.850325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d51d31-c68d-42ee-ab8e-18ac4bd6e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:11.881989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a2d51d31-c68d-42ee-ab8e-18ac4bd6e425 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:11.888153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a2d51d31-c68d-42ee-ab8e-18ac4bd6e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:11.893725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1007/2014] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:11.898620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3cd61-8c9f-4088-a949-e69a17c854ad None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:11.925498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3cd61-8c9f-4088-a949-e69a17c854ad None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:11.933180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:11.934046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:11.934377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:11.934707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:11.935587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:11.935961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:11.937692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:11.954416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1008/2015] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:02:11.958360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d51d31-c68d-42ee-ab8e-18ac4bd6e425 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:11.973222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d51d31-c68d-42ee-ab8e-18ac4bd6e425 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:11.979889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:11.979889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:11.980585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:11.980969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:11.981899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:11.983954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:12.059507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:12.081865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1008/2016] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:11 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:12.083397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:12.094529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324f189-0165-4fd1-a3be-4737d73efe7d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:12.174543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:12.175697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:12.176488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:12.176488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:12.177063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:12.179441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:12.389031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1009/2017] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:12 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:12.394583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:12.410039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a0e03-37a9-4633-a470-062ec1fcfbed admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:12.417662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:12.418614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:12.419111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:12.419841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:12.420734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:12.423090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:12.442636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1009/2018] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:12.443725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:12.458770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aac7b8-347a-4930-b1ed-0ef5c93e39c0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:12.473707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046aaa5-4cae-4b22-ba5d-8f00159a0a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:12.474445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046aaa5-4cae-4b22-ba5d-8f00159a0a2b admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:12.474993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046aaa5-4cae-4b22-ba5d-8f00159a0a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:12.475542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046aaa5-4cae-4b22-ba5d-8f00159a0a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:12.476819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046aaa5-4cae-4b22-ba5d-8f00159a0a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:12.479227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046aaa5-4cae-4b22-ba5d-8f00159a0a2b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:12.500913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1010/2019] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:12.501760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:12.542470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c191d2e-e4f2-45d5-a596-ce1817f43cf4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:12.542470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:12.542470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:12.542470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:12.542861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] PATH_INFO: `/v3/projects/db12b48c620d43c49c3e69cb5cc323a3/users/f034859c8a464f7190935bd37d15a3a7/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:12.566109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db12b48c620d43c49c3e69cb5cc323a3, user_id=f034859c8a464f7190935bd37d15a3a7, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:12.569005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:12.883716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1010/2020] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:12 2022] PUT /identity/v3/projects/db12b48c620d43c49c3e69cb5cc323a3/users/f034859c8a464f7190935bd37d15a3a7/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:12.894383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24379-a334-4f38-8116-d56d189a8759 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:12.895053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24379-a334-4f38-8116-d56d189a8759 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:12.895053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24379-a334-4f38-8116-d56d189a8759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:12.933027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3fc24379-a334-4f38-8116-d56d189a8759 None None] MFA Rules not processed for user `f034859c8a464f7190935bd37d15a3a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:12.935521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3fc24379-a334-4f38-8116-d56d189a8759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:13.077484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1011/2021] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:13.079401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:13.187944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525937d-1f1e-461a-9d03-710a3d9aa69d admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:13.202194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c799f-dbfc-4d8f-88a0-534662feeb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:13.203003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c799f-dbfc-4d8f-88a0-534662feeb83 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:13.203384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c799f-dbfc-4d8f-88a0-534662feeb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:13.203758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c799f-dbfc-4d8f-88a0-534662feeb83 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:13.204679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c799f-dbfc-4d8f-88a0-534662feeb83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:13.207213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c799f-dbfc-4d8f-88a0-534662feeb83 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:13.332084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1011/2022] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:13 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:13.336472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc24379-a334-4f38-8116-d56d189a8759 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:13.358250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc24379-a334-4f38-8116-d56d189a8759 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:13.383344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:13.431520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:13.434432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:13.435177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:13.490903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:13.491387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:13.491673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:13.491952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:13.493666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:13.496521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:13.510723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] MFA Rules not processed for user `55c90cee381347d9a2c4797a8ae8d580`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:13.511996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:13.623970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1012/2023] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:13.626040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:13.628411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:13.648641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa824352-42b6-4949-87aa-291fe2a03370 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:13.654785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:13.655549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:13.655946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:13.656322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:13.657084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:13.659339 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:13.689145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1012/2024] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:13 2022] POST /identity/v3/users => generated 298 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:13.698756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:13.710555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6feff-ce09-4192-9e28-7a1902782fc7 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:13.716343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:13.717129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:13.717502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:13.717873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:13.718623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:13.721088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:13.733960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1013/2025] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:13.738642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:13.750301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f2f7c-c75a-4479-89a4-d1dcdb2c5f43 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:13.755185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:13.755870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:13.756201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:13.756552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] PATH_INFO: `/v3/projects/ec2121a6b781466e81576e85d031b6fa/users/c80c47d968cd4a90beb2150316d936b3/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:13.774653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2121a6b781466e81576e85d031b6fa, user_id=c80c47d968cd4a90beb2150316d936b3, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:13.778675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:13.809964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1013/2026] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:13 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:13.814655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:13.910104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1014/2027] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:13 2022] PUT /identity/v3/projects/ec2121a6b781466e81576e85d031b6fa/users/c80c47d968cd4a90beb2150316d936b3/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:13.917306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:14.002393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f9f4a-d778-49bd-b7b2-be57501b5c80 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.020616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:14.064457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21672df5-9fbf-48c3-8653-e4cc2a723896 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.069652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.070335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.070623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.072927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.073574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.075285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.098426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1015/2028] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:14.103451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:14.117352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3f03-227e-4363-b6da-35479716c7a8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.123410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76fa75-3218-4f04-9285-b13754c3945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.124305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76fa75-3218-4f04-9285-b13754c3945d admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.125581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76fa75-3218-4f04-9285-b13754c3945d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.125581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76fa75-3218-4f04-9285-b13754c3945d admin admin] PATH_INFO: `/v3/projects/ec2121a6b781466e81576e85d031b6fa/users/c80c47d968cd4a90beb2150316d936b3/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.128588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76fa75-3218-4f04-9285-b13754c3945d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2121a6b781466e81576e85d031b6fa, user_id=c80c47d968cd4a90beb2150316d936b3, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.132075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76fa75-3218-4f04-9285-b13754c3945d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.138429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.139288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.139955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.140271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.141031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.144293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.145887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1016/2029] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:14 2022] PUT /identity/v3/projects/ec2121a6b781466e81576e85d031b6fa/users/c80c47d968cd4a90beb2150316d936b3/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:14.165992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.166274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.166740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.252793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] MFA Rules not processed for user `c80c47d968cd4a90beb2150316d936b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:14.257802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:14.258142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1014/2030] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:13 2022] POST /identity/v3/users => generated 293 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:14.260653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:14.301218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cebe8-bff2-4e79-8e59-98e91db05bf0 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.429298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1017/2031] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:14.441921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:14.444833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:14.454214 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.456351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.456351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.456351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.456800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.459265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.472477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1015/2032] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:14.477969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:14.489744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e450-3488-40b5-bd90-7ef5d62b18d8 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.499802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.500157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.500348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.501374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] PATH_INFO: `/v3/projects/3938bc7b824d4a25aedaeb9ca3268ca5/users/ccf70a559b2444e788e7ab195e6c2aaf/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.522924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3938bc7b824d4a25aedaeb9ca3268ca5, user_id=ccf70a559b2444e788e7ab195e6c2aaf, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.526366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.544874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69062e5-104c-420e-8b3c-08c2f9935d22 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.553083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.553993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.554379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.554795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.611774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1016/2033] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:14 2022] PUT /identity/v3/projects/3938bc7b824d4a25aedaeb9ca3268ca5/users/ccf70a559b2444e788e7ab195e6c2aaf/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:14.616547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:14.639722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.641505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.654514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1018/2034] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:14.688211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:14.709071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e4643-ab3b-4b0f-8884-6b014dfde22c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.715271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80124a4d-a9a2-4df9-847a-5a2955ea1b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.715953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80124a4d-a9a2-4df9-847a-5a2955ea1b96 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.716238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80124a4d-a9a2-4df9-847a-5a2955ea1b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.716511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80124a4d-a9a2-4df9-847a-5a2955ea1b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.717319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80124a4d-a9a2-4df9-847a-5a2955ea1b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.719429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80124a4d-a9a2-4df9-847a-5a2955ea1b96 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.747944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1017/2035] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:14.750983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:14.764194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa08415-5934-4045-8b4e-573a9334120f service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:14.769695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1c448-a00e-4eb8-9e02-3931d5ab71ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:14.770772 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1c448-a00e-4eb8-9e02-3931d5ab71ca admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:14.771129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1c448-a00e-4eb8-9e02-3931d5ab71ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:14.771802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1c448-a00e-4eb8-9e02-3931d5ab71ca admin admin] PATH_INFO: `/v3/projects/3938bc7b824d4a25aedaeb9ca3268ca5/users/ccf70a559b2444e788e7ab195e6c2aaf/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:14.782147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1c448-a00e-4eb8-9e02-3931d5ab71ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=3938bc7b824d4a25aedaeb9ca3268ca5, user_id=ccf70a559b2444e788e7ab195e6c2aaf, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:14.782147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1c448-a00e-4eb8-9e02-3931d5ab71ca admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:14.791724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1019/2036] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:14 2022] PUT /identity/v3/projects/3938bc7b824d4a25aedaeb9ca3268ca5/users/ccf70a559b2444e788e7ab195e6c2aaf/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:15.802923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:16.899129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:16.900190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:16.900190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:16.959636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] MFA Rules not processed for user `65c88d8b86544ab5825adbadb26b1e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:16.961754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:17.049642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1018/2037] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:17.062339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6f35c-d06c-4592-93b3-895ee0b757d5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:17.063123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6f35c-d06c-4592-93b3-895ee0b757d5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:17.063123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6f35c-d06c-4592-93b3-895ee0b757d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:17.084882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-56b6f35c-d06c-4592-93b3-895ee0b757d5 None None] MFA Rules not processed for user `65c88d8b86544ab5825adbadb26b1e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:17.086195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-56b6f35c-d06c-4592-93b3-895ee0b757d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:17.093583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1020/2038] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:17.107110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:17.108677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:17.224265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc574f2b-5e18-4e7d-b674-f7d1ad5bf7a7 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:17.230777 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:17.231414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:17.237826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:17.237826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:17.240997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:17.243353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:17.254233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1019/2039] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:17.344598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0916e7-d22e-47d5-9591-8ad2d145b969 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:17.344598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0916e7-d22e-47d5-9591-8ad2d145b969 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:17.344598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0916e7-d22e-47d5-9591-8ad2d145b969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:17.356611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6c0916e7-d22e-47d5-9591-8ad2d145b969 None None] MFA Rules not processed for user `ccf70a559b2444e788e7ab195e6c2aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:17.358046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6c0916e7-d22e-47d5-9591-8ad2d145b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:17.419863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:17.486862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1021/2040] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:17.498931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:17.608588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb50dc-af15-46d8-9e81-d030b2979c97 service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:17.608588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:17.608588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:17.608588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:17.608923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:17.668423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:17.670290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:17.696640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1020/2041] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:18.308896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:18.309407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:18.309500 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:18.317354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:18.349458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] MFA Rules not processed for user `65c88d8b86544ab5825adbadb26b1e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:18.351862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:18.395547 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1022/2042] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:19.559769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:19.561413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:19.670449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effefd92-55cf-4e45-ae58-7b60530d0090 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:19.676038 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:19.676706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:19.677180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:19.677448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] PATH_INFO: `/v3/users/efcb5a8c86e7406c9fad18c3126cfa26` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:19.679494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efcb5a8c86e7406c9fad18c3126cfa26)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:19.683451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:19.715508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:19.781837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:19.783435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:19.798073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d4efa-7534-4e02-aa8d-6657adfda069 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:19.802111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769545a-3261-4d17-a599-3e3791796ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:19.802735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769545a-3261-4d17-a599-3e3791796ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:19.803012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769545a-3261-4d17-a599-3e3791796ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:19.803287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769545a-3261-4d17-a599-3e3791796ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:19.822732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769545a-3261-4d17-a599-3e3791796ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:19.826319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769545a-3261-4d17-a599-3e3791796ad4 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:19.837049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1023/2043] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:19.986833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efcb5a8c86e7406c9fad18c3126cfa26'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:19.998241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efcb5a8c86e7406c9fad18c3126cfa26'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:20.126668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efcb5a8c86e7406c9fad18c3126cfa26'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:20.127448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efcb5a8c86e7406c9fad18c3126cfa26'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:20.194664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efcb5a8c86e7406c9fad18c3126cfa26'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:20.202888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1021/2044] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:19 2022] DELETE /identity/v3/users/efcb5a8c86e7406c9fad18c3126cfa26 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:20.207675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:20.207982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:20.208227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:20.239236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:20.240753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:20.381729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1024/2045] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:20.392658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:20.394308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:20.589532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573226-ebbc-490a-bc24-81458d4951e6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:20.603885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:20.605074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:20.606511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:20.606851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:20.628181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:20.633825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:20.645974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1022/2046] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:21.220325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:21.223081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:21.250083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d4207-9d54-4256-8a0a-f13484e116dc None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:21.269669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:21.269669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:21.269669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:21.269669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] PATH_INFO: `/v3/projects/307f777723bb4793aae69f9e494d135e` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:21.270171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=307f777723bb4793aae69f9e494d135e)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:21.270171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:21.514417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307f777723bb4793aae69f9e494d135e', 'request_id': 'req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:21.521570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307f777723bb4793aae69f9e494d135e', 'request_id': 'req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:21.528252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307f777723bb4793aae69f9e494d135e', 'request_id': 'req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:21.729118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] The token cache is being invalidate because project 307f777723bb4793aae69f9e494d135e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:21.737452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:21.737452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:21.737452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1025/2047] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:21 2022] DELETE /identity/v3/projects/307f777723bb4793aae69f9e494d135e => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:21.738567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:21.749324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661e313-a159-458f-8750-fe088eeef496 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:21.829454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:21.830169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:21.830455 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:21.830728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] PATH_INFO: `/v3/users/8706526ce3f64e31b9cd4feaa1f8c10a` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:21.832002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8706526ce3f64e31b9cd4feaa1f8c10a)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:21.834075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:22.134420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:22.146806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8706526ce3f64e31b9cd4feaa1f8c10a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:22.165012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8706526ce3f64e31b9cd4feaa1f8c10a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:22.255899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8706526ce3f64e31b9cd4feaa1f8c10a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:22.257229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8706526ce3f64e31b9cd4feaa1f8c10a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:22.283861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8706526ce3f64e31b9cd4feaa1f8c10a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:22.293544 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1023/2048] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:21 2022] DELETE /identity/v3/users/8706526ce3f64e31b9cd4feaa1f8c10a => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:22.363471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:22.365676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:22.465104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4fa5a-d476-4f0d-a853-493bf29a40d0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:22.491476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:22.561618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeb16d-ce48-40d9-963e-fa5e25a939bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:22.563087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeb16d-ce48-40d9-963e-fa5e25a939bf service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:22.564268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeb16d-ce48-40d9-963e-fa5e25a939bf service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:22.565808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeb16d-ce48-40d9-963e-fa5e25a939bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:22.572076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e2aeb16d-ce48-40d9-963e-fa5e25a939bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:22.684677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeb16d-ce48-40d9-963e-fa5e25a939bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:22.686346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeb16d-ce48-40d9-963e-fa5e25a939bf service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:22.709496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1026/2049] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:22.905560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:22.907212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:23.035032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46405bbc-ff2b-4531-9972-f9cbb76d83b1 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:23.040846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:23.041443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:23.041790 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:23.042051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] PATH_INFO: `/v3/projects/9ae1b1dd0cea4b64901600d370077fc2` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:23.043207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae1b1dd0cea4b64901600d370077fc2)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:23.045170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:23.201224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae1b1dd0cea4b64901600d370077fc2', 'request_id': 'req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:23.209416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae1b1dd0cea4b64901600d370077fc2', 'request_id': 'req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:23.362865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae1b1dd0cea4b64901600d370077fc2', 'request_id': 'req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:23.376603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] The token cache is being invalidate because project 9ae1b1dd0cea4b64901600d370077fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:23.377130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:23.377979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8c7b0fcf-50e7-4306-93eb-0f47c43a8eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:23.379607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1024/2050] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:22 2022] DELETE /identity/v3/projects/9ae1b1dd0cea4b64901600d370077fc2 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:23.386041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b4c8d-8c5f-4af1-ab8c-c7c1c42ea53b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:23.386609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b4c8d-8c5f-4af1-ab8c-c7c1c42ea53b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:23.386793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b4c8d-8c5f-4af1-ab8c-c7c1c42ea53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:23.410696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c58b4c8d-8c5f-4af1-ab8c-c7c1c42ea53b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:23.412745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c58b4c8d-8c5f-4af1-ab8c-c7c1c42ea53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:23.506043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1027/2051] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:23.516592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12213f2-d33b-4ccf-8503-d2aea4b8f342 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:23.517076 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12213f2-d33b-4ccf-8503-d2aea4b8f342 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:23.517651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12213f2-d33b-4ccf-8503-d2aea4b8f342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:23.552098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a12213f2-d33b-4ccf-8503-d2aea4b8f342 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:23.552098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a12213f2-d33b-4ccf-8503-d2aea4b8f342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:23.572106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b4c8d-8c5f-4af1-ab8c-c7c1c42ea53b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:23.584576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1025/2052] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:23.594084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b4c8d-8c5f-4af1-ab8c-c7c1c42ea53b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:23.617099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:23.624400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:23.625359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:23.625359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:23.625686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:23.626016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:23.634846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:23.669264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12213f2-d33b-4ccf-8503-d2aea4b8f342 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:23.671980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1028/2053] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:02:23.677038 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12213f2-d33b-4ccf-8503-d2aea4b8f342 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:23.686688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:23.688262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:23.688745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:23.689181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:23.690110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:23.692506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:23.876666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1026/2054] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:23 2022] POST /identity/v3/projects => generated 405 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:23.881343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:23.893915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09667c3d-b803-44b4-828c-6726077a90be admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:23.979595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:23.980222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:23.981895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:23.981895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:23.981895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:23.984017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:24.118484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1029/2055] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:23 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:24.124519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:24.135296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079e4a4-8a0f-4bde-a3e8-dd036c18aa2c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:24.146954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:24.147739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:24.149880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:24.149880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:24.152681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:24.161232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:24.172503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1027/2056] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:24.176938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:24.190945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2434cb-c840-4864-8008-e9177389b6ad admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:24.197345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76d4ee-46f2-40c8-b746-2726c2b0a055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:24.198606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d4ee-46f2-40c8-b746-2726c2b0a055 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:24.198909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d4ee-46f2-40c8-b746-2726c2b0a055 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:24.199185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d4ee-46f2-40c8-b746-2726c2b0a055 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:24.199908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76d4ee-46f2-40c8-b746-2726c2b0a055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:24.201649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76d4ee-46f2-40c8-b746-2726c2b0a055 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:24.215005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1030/2057] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:02:24.219713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:24.243675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b432d1-d133-4e16-8390-30f57627cd9b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:24.256810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:24.258124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:24.258785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:24.259393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] PATH_INFO: `/v3/projects/329ae09b6c264f3bbe539e111ef3f15d/users/4a5cab9c93084b47ad3fbbb4b1185bb0/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:24.266445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:24.305321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] RBAC: Authorizing `identity:create_grant(project_id=329ae09b6c264f3bbe539e111ef3f15d, user_id=4a5cab9c93084b47ad3fbbb4b1185bb0, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:24.308590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:24.493863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1028/2058] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:24 2022] PUT /identity/v3/projects/329ae09b6c264f3bbe539e111ef3f15d/users/4a5cab9c93084b47ad3fbbb4b1185bb0/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:24.506494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdc839-4186-4637-ad4e-5389c51c02c9 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:24.506494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdc839-4186-4637-ad4e-5389c51c02c9 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:24.507318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdc839-4186-4637-ad4e-5389c51c02c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:24.549144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3efdc839-4186-4637-ad4e-5389c51c02c9 None None] MFA Rules not processed for user `4a5cab9c93084b47ad3fbbb4b1185bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:24.550460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3efdc839-4186-4637-ad4e-5389c51c02c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:24.654800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1031/2059] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:24.659933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:24.660894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:24.774341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680648fb-e184-4897-ad2e-d5655641157a admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:24.779558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:24.780184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:24.780451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:24.780715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:24.781419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:24.783380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:24.913547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1029/2060] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:24 2022] POST /identity/v3/projects => generated 407 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:24.915624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdc839-4186-4637-ad4e-5389c51c02c9 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:24.936640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdc839-4186-4637-ad4e-5389c51c02c9 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:25.008861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:25.009483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:25.009761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:25.010057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:25.010690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:25.012619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:25.122055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1032/2061] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:24 2022] POST /identity/v3/users => generated 310 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:25.122699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:25.142264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1b4f-bb89-4048-ad55-2b426dddbe9e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:25.149809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:25.150448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:25.150726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:25.151068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:25.151731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:25.153673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:25.180322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:25.181009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1030/2062] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:02:25.194801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7f57-0f59-4f91-8136-b400b35fa230 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:25.205302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:25.206688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:25.206980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:25.207259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] PATH_INFO: `/v3/projects/efb355c014f147b6ac56e43fe8b59740/users/906a440400d847f49999bb82f8e88c69/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:25.224635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb355c014f147b6ac56e43fe8b59740, user_id=906a440400d847f49999bb82f8e88c69, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:25.227661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:25.355617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1033/2063] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:25 2022] PUT /identity/v3/projects/efb355c014f147b6ac56e43fe8b59740/users/906a440400d847f49999bb82f8e88c69/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:25.359916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:25.373023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835ca8-0b18-49c6-af6b-938b918159dd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:25.477706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af414ad-ae88-4f29-85f5-1bfd003ba2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:25.478354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af414ad-ae88-4f29-85f5-1bfd003ba2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:25.478718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af414ad-ae88-4f29-85f5-1bfd003ba2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:25.479159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af414ad-ae88-4f29-85f5-1bfd003ba2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:25.479823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af414ad-ae88-4f29-85f5-1bfd003ba2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:25.481510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af414ad-ae88-4f29-85f5-1bfd003ba2f7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:25.495042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1031/2064] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:25.499859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:25.511234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c800-8001-4fb6-b217-20a06a3c4a4f admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:25.516289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:25.516963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:25.517236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:25.517503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] PATH_INFO: `/v3/projects/efb355c014f147b6ac56e43fe8b59740/users/906a440400d847f49999bb82f8e88c69/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:25.521400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb355c014f147b6ac56e43fe8b59740, user_id=906a440400d847f49999bb82f8e88c69, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:25.524458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:25.536115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1034/2065] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:25 2022] PUT /identity/v3/projects/efb355c014f147b6ac56e43fe8b59740/users/906a440400d847f49999bb82f8e88c69/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:25.549561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:25.549969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:25.550318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:25.592267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] MFA Rules not processed for user `906a440400d847f49999bb82f8e88c69`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:25.594449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:25.681648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1032/2066] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:25 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:25.700347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:25.703960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:25.752886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:25.846893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7939e-3457-4c3e-95b7-bf3836eeef01 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:25.855463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:25.856487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:25.861421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:25.861421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:25.877083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:25.880662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:25.924131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1035/2067] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:25 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:26.196982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:26.555450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:26.558199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:26.580379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd605ac-4b0c-4b4a-98df-023cce4f7a28 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:26.586549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d4d3-62e6-4062-a563-f4879380ad72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:26.587960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d4d3-62e6-4062-a563-f4879380ad72 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:26.596978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d4d3-62e6-4062-a563-f4879380ad72 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:26.596978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d4d3-62e6-4062-a563-f4879380ad72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:26.665442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d4d3-62e6-4062-a563-f4879380ad72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:26.666952 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d4d3-62e6-4062-a563-f4879380ad72 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:26.677704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1033/2068] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:26 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:29.628727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:29.633490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:29.765281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230f916-c537-49f5-ac0a-05f6e255aaab service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:29.781148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:29.783140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:29.783140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:29.783140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] PATH_INFO: `/v3/users/f2c71c35684446d6b8bbc620506bd9c1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:29.784712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c71c35684446d6b8bbc620506bd9c1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:29.787706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:30.083845 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c71c35684446d6b8bbc620506bd9c1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:30.094863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c71c35684446d6b8bbc620506bd9c1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:30.096053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c71c35684446d6b8bbc620506bd9c1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:30.097528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c71c35684446d6b8bbc620506bd9c1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:30.190822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c71c35684446d6b8bbc620506bd9c1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:30.218879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1036/2069] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:29 2022] DELETE /identity/v3/users/f2c71c35684446d6b8bbc620506bd9c1 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:30.223875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:30.224411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:30.224411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:30.259257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:30.260396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:30.324521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1034/2070] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:30.334342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:30.335818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:30.384232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:30.397344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83329f31-c362-449e-b324-7a7fb7bd43d9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:30.401522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af9976-fc7c-463e-ab33-c71271eacf42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:30.402182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9976-fc7c-463e-ab33-c71271eacf42 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:30.402462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9976-fc7c-463e-ab33-c71271eacf42 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:30.402737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9976-fc7c-463e-ab33-c71271eacf42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:30.415262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af9976-fc7c-463e-ab33-c71271eacf42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:30.416748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af9976-fc7c-463e-ab33-c71271eacf42 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:30.428747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1037/2071] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:30.997089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:31.000798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:31.043775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda34f13-9459-46aa-b357-65de25acd306 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:31.054022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:31.055408 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:31.055953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:31.056507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] PATH_INFO: `/v3/projects/243a566c633744c3b1169a33d0764344` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:31.058178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243a566c633744c3b1169a33d0764344)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:31.060219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:31.261129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243a566c633744c3b1169a33d0764344', 'request_id': 'req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:31.268032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243a566c633744c3b1169a33d0764344', 'request_id': 'req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:31.396307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243a566c633744c3b1169a33d0764344', 'request_id': 'req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:31.416610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] The token cache is being invalidate because project 243a566c633744c3b1169a33d0764344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:31.417031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:31.418144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1aba9ffa-8fad-45d0-b9ab-8debd7f85fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:31.419710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1035/2072] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:30 2022] DELETE /identity/v3/projects/243a566c633744c3b1169a33d0764344 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:31.424956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30522299-b504-4cfa-876b-25cb4451e49e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:31.425749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30522299-b504-4cfa-876b-25cb4451e49e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:31.426222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30522299-b504-4cfa-876b-25cb4451e49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:31.458093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-30522299-b504-4cfa-876b-25cb4451e49e None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:31.459431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-30522299-b504-4cfa-876b-25cb4451e49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:31.547886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1038/2073] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:31.562504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a65f0e-dc57-4384-9ec8-f1a1c5573263 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:31.562776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a65f0e-dc57-4384-9ec8-f1a1c5573263 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:31.563055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a65f0e-dc57-4384-9ec8-f1a1c5573263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:31.571467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:31.586748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f0a65f0e-dc57-4384-9ec8-f1a1c5573263 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:31.591624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f0a65f0e-dc57-4384-9ec8-f1a1c5573263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:31.597115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1036/2074] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:31.601946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30522299-b504-4cfa-876b-25cb4451e49e None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:31.614498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30522299-b504-4cfa-876b-25cb4451e49e None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:31.619768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:31.620380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:31.620656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:31.621246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:31.621866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:31.622172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:31.623784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:31.636066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1039/2075] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:02:31.640610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a65f0e-dc57-4384-9ec8-f1a1c5573263 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:31.651117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a65f0e-dc57-4384-9ec8-f1a1c5573263 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:31.657929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:31.660247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:31.660247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:31.660247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:31.660247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:31.661695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:31.771972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1037/2076] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:31 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:31.775589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:31.786162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75427e4d-f89c-40db-8b22-811476f7b176 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:31.864386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:31.865030 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:31.865311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:31.865593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:31.866499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:31.869228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:31.985600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1040/2077] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:31 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:31.993278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.005128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d25b4-bc86-4619-8cc9-837c40eec689 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.010363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.010975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.011258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.011530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.012188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.013798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:32.028451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1038/2078] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:32.031522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.042703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf28887-1b4b-4ea6-a0cc-3731316a72f2 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.048054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97147daf-81e6-4e06-887c-848958688875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.048874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97147daf-81e6-4e06-887c-848958688875 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.049249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97147daf-81e6-4e06-887c-848958688875 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.049619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97147daf-81e6-4e06-887c-848958688875 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.050512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97147daf-81e6-4e06-887c-848958688875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.052371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97147daf-81e6-4e06-887c-848958688875 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:32.065111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1041/2079] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:32.069159 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.080689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9af542-e461-4c3c-a0e8-24a2de0471ac admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.086687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.087093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.087356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.087598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] PATH_INFO: `/v3/projects/a94f295b153345069394ea7a51c56313/users/6e020d7962994c70b79552c78a34a905/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.110919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94f295b153345069394ea7a51c56313, user_id=6e020d7962994c70b79552c78a34a905, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.114880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:32.221087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1039/2080] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:32 2022] PUT /identity/v3/projects/a94f295b153345069394ea7a51c56313/users/6e020d7962994c70b79552c78a34a905/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:32.232918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9eb12-1190-4d0c-8e60-38200b123b14 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.233266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9eb12-1190-4d0c-8e60-38200b123b14 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.233574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9eb12-1190-4d0c-8e60-38200b123b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.277628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-58f9eb12-1190-4d0c-8e60-38200b123b14 None None] MFA Rules not processed for user `6e020d7962994c70b79552c78a34a905`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:32.280911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-58f9eb12-1190-4d0c-8e60-38200b123b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:32.353219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1042/2081] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:32.358536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.431403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015cc32-cf49-439e-a6ed-75c02461d78b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.436883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.437502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.437804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.438074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.438721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.440689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:32.540434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1040/2082] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:32 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:32.541227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9eb12-1190-4d0c-8e60-38200b123b14 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.559495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9eb12-1190-4d0c-8e60-38200b123b14 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.655382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.656084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.658253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.658807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.659712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.662488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:32.797716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1043/2083] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:32 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:32.802289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.815973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee389dd-76e2-4880-9f6c-5bb19b5570c0 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.821874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.822613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.822941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.823209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.823867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.825526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:32.841486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1041/2084] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:32.842959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.854858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f014d1-a212-4f68-9828-ea2d134bf2e5 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.860619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f081a-0cf4-43b2-b2b0-e4056f834c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.861466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f081a-0cf4-43b2-b2b0-e4056f834c3a admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.862006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f081a-0cf4-43b2-b2b0-e4056f834c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.862415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f081a-0cf4-43b2-b2b0-e4056f834c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.863339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f081a-0cf4-43b2-b2b0-e4056f834c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.865312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f081a-0cf4-43b2-b2b0-e4056f834c3a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:32.877502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1044/2085] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:32.883145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:32.901715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df633f44-6dc0-42e6-a108-204eb246a874 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:32.908870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:32.910084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:32.910578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:32.911082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] PATH_INFO: `/v3/projects/2703b25174b14d70a496725e5f00fea9/users/9c4e6fde50f14e9482cc72d36ad1faeb/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:32.939008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2703b25174b14d70a496725e5f00fea9, user_id=9c4e6fde50f14e9482cc72d36ad1faeb, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:32.944584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:33.076918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1042/2086] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:32 2022] PUT /identity/v3/projects/2703b25174b14d70a496725e5f00fea9/users/9c4e6fde50f14e9482cc72d36ad1faeb/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:33.088257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff37ac-2c19-43ba-a8dc-818853b1a59d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.088588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff37ac-2c19-43ba-a8dc-818853b1a59d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.088961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff37ac-2c19-43ba-a8dc-818853b1a59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.117514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-a4ff37ac-2c19-43ba-a8dc-818853b1a59d None None] MFA Rules not processed for user `9c4e6fde50f14e9482cc72d36ad1faeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:33.118835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a4ff37ac-2c19-43ba-a8dc-818853b1a59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:33.185202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1045/2087] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:33.189913 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:33.254853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2734e-92b2-405c-881f-8c6af3186bdc admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:33.260532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:33.261145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.261474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.261696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.262467 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:33.264409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:33.379208 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1043/2088] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:33 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:33.386605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff37ac-2c19-43ba-a8dc-818853b1a59d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:33.404304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff37ac-2c19-43ba-a8dc-818853b1a59d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:33.474095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:33.474712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.474987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.475286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.475932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:33.477910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:33.571933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1046/2089] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:33 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:33.573283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:33.595839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4b997-1af6-4e6e-9c62-9492a22e4378 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:33.621610 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:33.622349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.622759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.623110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.624042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:33.625923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:33.645416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1044/2090] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:33.650800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:33.664361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0e72f-d09a-47f9-941f-7d3df03c3e89 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:33.671484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:33.672357 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.672645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.672951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] PATH_INFO: `/v3/projects/e7afae0aec2c4243a04f9e12c5b9b6b6/users/568a02129f52424c9b54c4829d8136ce/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.700306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7afae0aec2c4243a04f9e12c5b9b6b6, user_id=568a02129f52424c9b54c4829d8136ce, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:33.703718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:33.790543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1047/2091] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:33 2022] PUT /identity/v3/projects/e7afae0aec2c4243a04f9e12c5b9b6b6/users/568a02129f52424c9b54c4829d8136ce/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:33.795747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:33.806607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f927a3-ddeb-494c-af36-457518f27cef admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:33.867284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2066da0-6235-486c-82a8-2f47da0d43f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:33.867892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066da0-6235-486c-82a8-2f47da0d43f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.868245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066da0-6235-486c-82a8-2f47da0d43f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.868516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066da0-6235-486c-82a8-2f47da0d43f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.869186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2066da0-6235-486c-82a8-2f47da0d43f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:33.870788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2066da0-6235-486c-82a8-2f47da0d43f7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:33.881679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1045/2092] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:33.886937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:33.897671 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e183-a83f-4c45-b950-1943e387cb5e admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:33.902924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:33.903567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.903841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.904113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] PATH_INFO: `/v3/projects/e7afae0aec2c4243a04f9e12c5b9b6b6/users/568a02129f52424c9b54c4829d8136ce/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.906071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7afae0aec2c4243a04f9e12c5b9b6b6, user_id=568a02129f52424c9b54c4829d8136ce, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:33.908878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:33.919336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1048/2093] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:33 2022] PUT /identity/v3/projects/e7afae0aec2c4243a04f9e12c5b9b6b6/users/568a02129f52424c9b54c4829d8136ce/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:33.930803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:33.931090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:33.931437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:33.959896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] MFA Rules not processed for user `568a02129f52424c9b54c4829d8136ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:33.961587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:34.048979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1046/2094] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:34.060904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:34.062627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:34.077631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:34.123004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea29300-bf60-4b07-9497-5a63c494fd4e admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:34.127008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf0e3e-8c17-40c7-ac23-8899f4b4c934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:34.127629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf0e3e-8c17-40c7-ac23-8899f4b4c934 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:34.127903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf0e3e-8c17-40c7-ac23-8899f4b4c934 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:34.128176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf0e3e-8c17-40c7-ac23-8899f4b4c934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:34.186968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf0e3e-8c17-40c7-ac23-8899f4b4c934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:34.188407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf0e3e-8c17-40c7-ac23-8899f4b4c934 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:34.197976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1049/2095] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:34.436180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:35.362173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:36.023964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:36.025441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:36.128242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:36.152665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94929e3e-ef73-4522-a889-bd5100e6af38 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:36.166859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:36.166859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:36.166859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:36.166859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] PATH_INFO: `/v3/users/f034859c8a464f7190935bd37d15a3a7` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:36.167225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f034859c8a464f7190935bd37d15a3a7)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:36.167225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:36.559873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f034859c8a464f7190935bd37d15a3a7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:36.571579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f034859c8a464f7190935bd37d15a3a7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:36.647393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f034859c8a464f7190935bd37d15a3a7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:36.649187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f034859c8a464f7190935bd37d15a3a7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:36.663523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f034859c8a464f7190935bd37d15a3a7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:36.669578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1047/2096] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:36 2022] DELETE /identity/v3/users/f034859c8a464f7190935bd37d15a3a7 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:36.674097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:36.674887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:36.674887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:36.694160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:36.695912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:36.761221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:36.773908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1050/2097] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:36.786309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:36.788511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:36.883945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d6a61-0625-4d7e-aadb-4ec0ad64d427 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:36.887650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:36.888377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:36.888661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:36.889002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:36.900948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:36.902687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:36.913262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1048/2098] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:37.334396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:37.336726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:37.412130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86640681-56bc-4513-8a8f-e7e8821a9d00 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:37.415894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:37.416522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:37.416870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:37.417118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:37.418921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:37.479094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:37.481179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:37.483106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:37.484579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:37.497700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1051/2099] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:37.550716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19190c-772c-4aa4-a166-d2c78f368bd5 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:37.550716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:37.550716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:37.550716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:37.552147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] PATH_INFO: `/v3/projects/db12b48c620d43c49c3e69cb5cc323a3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:37.552147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=db12b48c620d43c49c3e69cb5cc323a3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:37.552147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:37.689835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db12b48c620d43c49c3e69cb5cc323a3', 'request_id': 'req-d68f0754-264e-4d97-9ecf-c4af0350c3bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:37.697588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db12b48c620d43c49c3e69cb5cc323a3', 'request_id': 'req-d68f0754-264e-4d97-9ecf-c4af0350c3bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:37.702769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:37.841751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db12b48c620d43c49c3e69cb5cc323a3', 'request_id': 'req-d68f0754-264e-4d97-9ecf-c4af0350c3bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:37.859652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] The token cache is being invalidate because project db12b48c620d43c49c3e69cb5cc323a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:37.860124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:37.861155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:37.863080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1049/2100] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:37 2022] DELETE /identity/v3/projects/db12b48c620d43c49c3e69cb5cc323a3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:37.868168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:37.885346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983319f4-7fc7-4498-b4e3-79325823d506 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:37.951140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:37.951788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:37.952069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:37.952371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] PATH_INFO: `/v3/users/c80c47d968cd4a90beb2150316d936b3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:37.953535 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80c47d968cd4a90beb2150316d936b3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:37.956742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:38.264717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80c47d968cd4a90beb2150316d936b3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:38.270764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:38.274457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80c47d968cd4a90beb2150316d936b3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:38.275366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80c47d968cd4a90beb2150316d936b3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:38.276015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80c47d968cd4a90beb2150316d936b3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:38.447802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3f3cef87-52c7-44cd-bc6c-f03143904e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80c47d968cd4a90beb2150316d936b3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:38.473853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1052/2101] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:37 2022] DELETE /identity/v3/users/c80c47d968cd4a90beb2150316d936b3 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:39.060496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:39.062171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:39.077761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f0754-264e-4d97-9ecf-c4af0350c3bf admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:39.145671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:39.146909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:39.147211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:39.147743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] PATH_INFO: `/v3/projects/ec2121a6b781466e81576e85d031b6fa` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:39.149363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2121a6b781466e81576e85d031b6fa)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:39.151379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:39.330815 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2121a6b781466e81576e85d031b6fa', 'request_id': 'req-78f1c4dd-cc15-4435-99a8-3757bf772b78'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:39.336217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2121a6b781466e81576e85d031b6fa', 'request_id': 'req-78f1c4dd-cc15-4435-99a8-3757bf772b78'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:39.489754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2121a6b781466e81576e85d031b6fa', 'request_id': 'req-78f1c4dd-cc15-4435-99a8-3757bf772b78'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:39.524541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] The token cache is being invalidate because project ec2121a6b781466e81576e85d031b6fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:39.524920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:39.525726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:39.528410 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1050/2102] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:39 2022] DELETE /identity/v3/projects/ec2121a6b781466e81576e85d031b6fa => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:39.534602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:39.535413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:39.535689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:39.542185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:39.559513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:39.563434 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:39.564918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:39.648224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1c4dd-cc15-4435-99a8-3757bf772b78 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:39.654455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1053/2103] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:39.656000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:39.657538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:39.666418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:39.672386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:39.672386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:39.672386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] PATH_INFO: `/v3/users/4a5cab9c93084b47ad3fbbb4b1185bb0` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:39.672386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a5cab9c93084b47ad3fbbb4b1185bb0)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:39.672386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:39.700436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:39.736068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b5615-e5ff-4006-bd69-14add9624c71 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:39.740983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b0e09-f43e-44f3-a6d3-4267bf32c51b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:39.741643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b0e09-f43e-44f3-a6d3-4267bf32c51b service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:39.742445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b0e09-f43e-44f3-a6d3-4267bf32c51b service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:39.742722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b0e09-f43e-44f3-a6d3-4267bf32c51b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:39.744542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-745b0e09-f43e-44f3-a6d3-4267bf32c51b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:39.821882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b0e09-f43e-44f3-a6d3-4267bf32c51b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:39.823365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b0e09-f43e-44f3-a6d3-4267bf32c51b service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:39.834518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1054/2104] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:39.835848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfdb2-cf9d-47d2-b76e-8d87a5956606 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:39.836241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfdb2-cf9d-47d2-b76e-8d87a5956606 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:39.836524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfdb2-cf9d-47d2-b76e-8d87a5956606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:39.874061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-732bfdb2-cf9d-47d2-b76e-8d87a5956606 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:39.875475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-732bfdb2-cf9d-47d2-b76e-8d87a5956606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:39.888121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1055/2105] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:39.888323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a5cab9c93084b47ad3fbbb4b1185bb0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:39.889961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bfdb2-cf9d-47d2-b76e-8d87a5956606 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:39.893403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a5cab9c93084b47ad3fbbb4b1185bb0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:39.902002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bfdb2-cf9d-47d2-b76e-8d87a5956606 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:39.908621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:39.908621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:39.909126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:39.909288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:39.910071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:39.910381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:39.912378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:39.924456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:39.931830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1056/2106] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:02:39.933332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:39.943425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f6a11-916b-4bfe-85a3-cdb877380226 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:39.948315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:39.948930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:39.949203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:39.949506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:39.950169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:39.952098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:40.036146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a5cab9c93084b47ad3fbbb4b1185bb0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:40.036923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a5cab9c93084b47ad3fbbb4b1185bb0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:40.053867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-183ae5a7-84bc-4da5-8806-cff41acc39ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a5cab9c93084b47ad3fbbb4b1185bb0'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:40.066945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1051/2107] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:39 2022] DELETE /identity/v3/users/4a5cab9c93084b47ad3fbbb4b1185bb0 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:40.069080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8558765-841b-481d-9890-4a59f103196d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.069080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8558765-841b-481d-9890-4a59f103196d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.069080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8558765-841b-481d-9890-4a59f103196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.078963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1057/2108] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:39 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:40.086004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:40.087868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:40.092321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b8558765-841b-481d-9890-4a59f103196d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:40.093634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b8558765-841b-481d-9890-4a59f103196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:40.160171 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1052/2109] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:40.162250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999aa82-8a66-4858-aa45-e0d41d6b66be admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:40.168119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:40.168776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.169152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.169559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.170240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:40.170759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8558765-841b-481d-9890-4a59f103196d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:40.172168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b8558765-841b-481d-9890-4a59f103196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:40.172294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:40.242521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8558765-841b-481d-9890-4a59f103196d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:40.246712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:40.247726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.247726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.247726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.258824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:40.260539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:40.271695 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1058/2110] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:40 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:40.272464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1053/2111] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:40.281334 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:40.292789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eeb2e0-6a4b-41f3-9add-4a676502e08b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:40.298804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:40.299419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.299694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.299970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.300756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:40.302392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:40.314153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1059/2112] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:40.318123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:40.332532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a68103-3960-43e9-b2e2-211d9f6bc6a8 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:40.341452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f1edc-8467-40c3-94a1-e7ec0a774c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:40.342164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f1edc-8467-40c3-94a1-e7ec0a774c62 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.342727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f1edc-8467-40c3-94a1-e7ec0a774c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.343186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f1edc-8467-40c3-94a1-e7ec0a774c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.343930 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f1edc-8467-40c3-94a1-e7ec0a774c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:40.345646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f1edc-8467-40c3-94a1-e7ec0a774c62 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:40.361935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1054/2113] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:40.362123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:40.373727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a5b5-330d-45e7-b8bb-78b57d53f600 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:40.379223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:40.379900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.380203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.380478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] PATH_INFO: `/v3/projects/4e5e9f2fb4484bd991093dc7cb53bb40/users/61ad97bfd1bf46fa854ab0fd69feb663/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.401623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5e9f2fb4484bd991093dc7cb53bb40, user_id=61ad97bfd1bf46fa854ab0fd69feb663, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:40.404893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:40.517360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1060/2114] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:40 2022] PUT /identity/v3/projects/4e5e9f2fb4484bd991093dc7cb53bb40/users/61ad97bfd1bf46fa854ab0fd69feb663/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:40.535962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.536304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.536646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.569055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] MFA Rules not processed for user `61ad97bfd1bf46fa854ab0fd69feb663`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:40.570928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:40.647870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1055/2115] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:40.660115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:40.662517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:40.755819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b1ec4-4a74-44bd-9012-315b8c040a35 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:40.760229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:40.760979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.761269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.761546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.770753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:40.772413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:40.793058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1061/2116] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:40.818925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:40.818925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:40.900561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:40.920778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94855e-28af-444d-af44-6b8c13c7103b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:40.920778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:40.920778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:40.920778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:40.921173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] PATH_INFO: `/v3/projects/329ae09b6c264f3bbe539e111ef3f15d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:40.921173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=329ae09b6c264f3bbe539e111ef3f15d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:40.921173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:41.032927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329ae09b6c264f3bbe539e111ef3f15d', 'request_id': 'req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.039724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329ae09b6c264f3bbe539e111ef3f15d', 'request_id': 'req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.167279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329ae09b6c264f3bbe539e111ef3f15d', 'request_id': 'req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.181340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] The token cache is being invalidate because project 329ae09b6c264f3bbe539e111ef3f15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:41.181634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.182372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.183916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1056/2117] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:40 2022] DELETE /identity/v3/projects/329ae09b6c264f3bbe539e111ef3f15d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:41.188201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:41.199986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40842d3-ad19-438a-ad22-0fd96d3112bb service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:41.236891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:41.271652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:41.272283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:41.272560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:41.272868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] PATH_INFO: `/v3/users/906a440400d847f49999bb82f8e88c69` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:41.280384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=906a440400d847f49999bb82f8e88c69)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:41.281754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:41.615187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906a440400d847f49999bb82f8e88c69'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.623059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906a440400d847f49999bb82f8e88c69'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.623978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906a440400d847f49999bb82f8e88c69'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.624819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906a440400d847f49999bb82f8e88c69'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.756590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-092dcb95-6435-465f-89c8-0cc83d1b4fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906a440400d847f49999bb82f8e88c69'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:41.796967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1062/2118] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:41 2022] DELETE /identity/v3/users/906a440400d847f49999bb82f8e88c69 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:42.446827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:42.448602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:42.469238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23d680-e551-4e92-a1da-cc6d42cb3ebe admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:42.540436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:42.541292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:42.541485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:42.541687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] PATH_INFO: `/v3/projects/efb355c014f147b6ac56e43fe8b59740` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:42.542928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb355c014f147b6ac56e43fe8b59740)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:42.544845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:42.677139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb355c014f147b6ac56e43fe8b59740', 'request_id': 'req-d755a891-2448-44e4-ac82-8f7ab17d409f'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:42.683491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb355c014f147b6ac56e43fe8b59740', 'request_id': 'req-d755a891-2448-44e4-ac82-8f7ab17d409f'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:42.789756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:42.790725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb355c014f147b6ac56e43fe8b59740', 'request_id': 'req-d755a891-2448-44e4-ac82-8f7ab17d409f'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:42.815980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] The token cache is being invalidate because project efb355c014f147b6ac56e43fe8b59740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:42.818142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:42.820261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d755a891-2448-44e4-ac82-8f7ab17d409f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:42.822050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1057/2119] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:42 2022] DELETE /identity/v3/projects/efb355c014f147b6ac56e43fe8b59740 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:42.827103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d181-da01-432b-8dc0-73526cacc9b4 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:42.827559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d181-da01-432b-8dc0-73526cacc9b4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:42.827638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d181-da01-432b-8dc0-73526cacc9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:42.850042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5593d181-da01-432b-8dc0-73526cacc9b4 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:42.852101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5593d181-da01-432b-8dc0-73526cacc9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:42.954723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1063/2120] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:42.963635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d529-318d-4b21-bf82-dc7b4b11759b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:42.964125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d529-318d-4b21-bf82-dc7b4b11759b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:42.964473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d529-318d-4b21-bf82-dc7b4b11759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.001262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d1c1d529-318d-4b21-bf82-dc7b4b11759b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:43.002681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d1c1d529-318d-4b21-bf82-dc7b4b11759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:43.012211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1058/2121] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:43.018195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593d181-da01-432b-8dc0-73526cacc9b4 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:43.027722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593d181-da01-432b-8dc0-73526cacc9b4 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:43.032401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:43.033001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:43.033275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:43.033543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.034534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:43.046558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:43.046558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:43.054338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1064/2122] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:02:43.058781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d529-318d-4b21-bf82-dc7b4b11759b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:43.068522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d529-318d-4b21-bf82-dc7b4b11759b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:43.073243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:43.073875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:43.074156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:43.074427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.075097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:43.077099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:43.272119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1059/2123] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:43 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:43.407069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:43.417032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ffbc4-1ee9-4b44-90b2-d230baf96ec8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:43.470198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:43.494256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:43.494919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:43.495210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:43.495493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.496390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:43.498347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:43.614800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1065/2124] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:43 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:43.620245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:43.631779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ae484-879c-48c8-9d73-f978b044ff80 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:43.644620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:43.644620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:43.644620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:43.644620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.645065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:43.645065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:43.654856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1060/2125] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:43.660901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:43.672894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3d293-64a6-4fe8-8f66-6cdc3f032dac admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:43.678596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f919-3992-40b9-bb7a-b77ef1327f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:43.679928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f919-3992-40b9-bb7a-b77ef1327f5c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:43.680405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f919-3992-40b9-bb7a-b77ef1327f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:43.680913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f919-3992-40b9-bb7a-b77ef1327f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.681570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f919-3992-40b9-bb7a-b77ef1327f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:43.683225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f919-3992-40b9-bb7a-b77ef1327f5c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:43.694410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1066/2126] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:43.699340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:43.709648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6dff2-b89f-4fd8-9949-00464702e2dc admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:43.714325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:43.719450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:43.719450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:43.719450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] PATH_INFO: `/v3/projects/596cdff155be4dc997ce351b7cd6db08/users/9c60f6eaccad440abff863fe99b40382/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.732152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=596cdff155be4dc997ce351b7cd6db08, user_id=9c60f6eaccad440abff863fe99b40382, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:43.735054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:43.851247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1061/2127] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:43 2022] PUT /identity/v3/projects/596cdff155be4dc997ce351b7cd6db08/users/9c60f6eaccad440abff863fe99b40382/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:43.856830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:43.856830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:43.856830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:43.882892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:43.890262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:43.956953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1067/2128] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:43.967439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:43.969391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:44.031419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e116e-7ab1-4e26-9e6c-52a7e611ae5c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:44.035132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:44.035802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:44.036082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:44.036354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:44.046910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:44.048352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:44.060257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1062/2129] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:45.330176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:45.334053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:45.347803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111b097-4d4d-4344-845a-ab06b14a5026 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:45.354099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:45.355043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:45.355484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:45.355932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] PATH_INFO: `/v3/users/61ad97bfd1bf46fa854ab0fd69feb663` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:45.357119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ad97bfd1bf46fa854ab0fd69feb663)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:45.359480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:45.377212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:45.384392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:45.419696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e70a78-4b8d-4440-8c02-c10c1686ec68 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:45.427780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:45.429366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:45.429852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:45.430298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] PATH_INFO: `/v3/users/6e020d7962994c70b79552c78a34a905` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:45.431515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e020d7962994c70b79552c78a34a905)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:45.433451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:45.572370 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ad97bfd1bf46fa854ab0fd69feb663'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.581730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ad97bfd1bf46fa854ab0fd69feb663'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.584065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ad97bfd1bf46fa854ab0fd69feb663'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.584065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ad97bfd1bf46fa854ab0fd69feb663'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.652035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e020d7962994c70b79552c78a34a905'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.659718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e020d7962994c70b79552c78a34a905'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.682283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7ceb4be-f5af-4a68-8f04-73cb9acc2313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ad97bfd1bf46fa854ab0fd69feb663'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.715689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1068/2130] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:45 2022] DELETE /identity/v3/users/61ad97bfd1bf46fa854ab0fd69feb663 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:45.720387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:45.720768 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:45.721222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:45.749381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:45.749381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:45.761876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:45.790647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e020d7962994c70b79552c78a34a905'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.792097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e020d7962994c70b79552c78a34a905'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.817380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5c8e91a3-0b71-496e-8df9-906a0e72bda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e020d7962994c70b79552c78a34a905'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:45.839488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1063/2131] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:45 2022] DELETE /identity/v3/users/6e020d7962994c70b79552c78a34a905 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:45.844591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:45.848162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:45.848310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1069/2132] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:45.848392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:45.870157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:45.873516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:45.896156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:45.898079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:45.977659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3d261-bd2d-4bba-ae2d-c039f08c7f6c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:45.982891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1064/2133] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:45.984004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:45.985377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:45.999047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18053e-7ca8-4db6-8798-9d963000c20d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:46.003827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:46.004463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:46.004741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:46.005147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] PATH_INFO: `/v3/users/55c90cee381347d9a2c4797a8ae8d580` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:46.006321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c90cee381347d9a2c4797a8ae8d580)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:46.008376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:46.035897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00c546-5ef2-44a6-9953-9e17314b6fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:46.036516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00c546-5ef2-44a6-9953-9e17314b6fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:46.036929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00c546-5ef2-44a6-9953-9e17314b6fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:46.037208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00c546-5ef2-44a6-9953-9e17314b6fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:46.046824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:46.049963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00c546-5ef2-44a6-9953-9e17314b6fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:46.051496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00c546-5ef2-44a6-9953-9e17314b6fc3 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:46.061827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1070/2134] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:46.063931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:46.064285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:46.064655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:46.093679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] MFA Rules not processed for user `9c60f6eaccad440abff863fe99b40382`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:46.094943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:46.163925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1071/2135] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:46.171533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:46.171533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:46.176183 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c90cee381347d9a2c4797a8ae8d580'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.179969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd50e0-b66f-4933-8642-360dcd4ef9d5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:46.183439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c90cee381347d9a2c4797a8ae8d580'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.183867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:46.184463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:46.184737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:46.185006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:46.193981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:46.195469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:46.205139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1072/2136] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:46.206112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:46.211623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:46.267684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d6a32-e1a0-4064-a42f-f5a639b3d1e8 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:46.271847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:46.280865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:46.280865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:46.280865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:46.283875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:46.285340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:46.295320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1073/2137] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:46.320043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c90cee381347d9a2c4797a8ae8d580'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.320043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c90cee381347d9a2c4797a8ae8d580'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.337001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c90cee381347d9a2c4797a8ae8d580'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.350853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1065/2138] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:45 2022] DELETE /identity/v3/users/55c90cee381347d9a2c4797a8ae8d580 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:46.558379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:46.560408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:46.579358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bdfef-2a08-4be6-993d-40c2f6421ab1 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:46.665022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:46.665702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:46.666018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:46.666294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] PATH_INFO: `/v3/projects/4e5e9f2fb4484bd991093dc7cb53bb40` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:46.667469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e5e9f2fb4484bd991093dc7cb53bb40)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:46.669466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:46.680488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:46.682568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:46.707480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfe001-22b7-4678-8afe-1ba78aad816b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:46.713615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:46.726748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:46.727435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:46.727724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] PATH_INFO: `/v3/projects/d7387ec4a8da43d39561bb4d6f5d11ad` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:46.729219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7387ec4a8da43d39561bb4d6f5d11ad)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:46.731232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:46.836409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e5e9f2fb4484bd991093dc7cb53bb40', 'request_id': 'req-bf4f9588-156a-4f7e-a1bd-6faccc54308e'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.842997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e5e9f2fb4484bd991093dc7cb53bb40', 'request_id': 'req-bf4f9588-156a-4f7e-a1bd-6faccc54308e'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.856189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e5e9f2fb4484bd991093dc7cb53bb40', 'request_id': 'req-bf4f9588-156a-4f7e-a1bd-6faccc54308e'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.859373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7387ec4a8da43d39561bb4d6f5d11ad', 'request_id': 'req-87c21c56-3b57-4ccd-8182-cda4b12b78c3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.864876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7387ec4a8da43d39561bb4d6f5d11ad', 'request_id': 'req-87c21c56-3b57-4ccd-8182-cda4b12b78c3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.967249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] The token cache is being invalidate because project 4e5e9f2fb4484bd991093dc7cb53bb40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:46.970410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.970410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:46.970410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1074/2139] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:46 2022] DELETE /identity/v3/projects/4e5e9f2fb4484bd991093dc7cb53bb40 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:46.970903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:46.971985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:47.067953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7387ec4a8da43d39561bb4d6f5d11ad', 'request_id': 'req-87c21c56-3b57-4ccd-8182-cda4b12b78c3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.087713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] The token cache is being invalidate because project d7387ec4a8da43d39561bb4d6f5d11ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:47.089936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.090327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f9588-156a-4f7e-a1bd-6faccc54308e admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:47.092571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-87c21c56-3b57-4ccd-8182-cda4b12b78c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.096199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1066/2140] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:46 2022] DELETE /identity/v3/projects/d7387ec4a8da43d39561bb4d6f5d11ad => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:47.100363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.101983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.102668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.130152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:47.131644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:47.164218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:47.165219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.165219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.165386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] PATH_INFO: `/v3/projects/a94f295b153345069394ea7a51c56313` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.166628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94f295b153345069394ea7a51c56313)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:47.168563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:47.233101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1067/2141] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:47.239804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:47.241158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:47.255842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd606c7d-82d4-46cc-bdbb-90c1c90b8e23 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:47.260151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:47.261555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.262112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.262112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] PATH_INFO: `/v3/users/ccf70a559b2444e788e7ab195e6c2aaf` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.263762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf70a559b2444e788e7ab195e6c2aaf)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:47.265642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:47.373569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94f295b153345069394ea7a51c56313', 'request_id': 'req-49f752d7-61d6-4910-9b76-1b952b544ed1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.380139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94f295b153345069394ea7a51c56313', 'request_id': 'req-49f752d7-61d6-4910-9b76-1b952b544ed1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.387235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94f295b153345069394ea7a51c56313', 'request_id': 'req-49f752d7-61d6-4910-9b76-1b952b544ed1'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.503786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] The token cache is being invalidate because project a94f295b153345069394ea7a51c56313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:47.504078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.504727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-49f752d7-61d6-4910-9b76-1b952b544ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.506273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1075/2142] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:46 2022] DELETE /identity/v3/projects/a94f295b153345069394ea7a51c56313 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:47.507515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.507788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.508056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.515693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf70a559b2444e788e7ab195e6c2aaf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.523327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf70a559b2444e788e7ab195e6c2aaf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.527758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:47.529001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:47.595258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1076/2143] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:47.596324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:47.597660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:47.609966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79153dfd-5714-48d3-b1cd-5b65ccf2d071 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:47.614535 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:47.615191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.615467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.615735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] PATH_INFO: `/v3/users/9c4e6fde50f14e9482cc72d36ad1faeb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.616917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4e6fde50f14e9482cc72d36ad1faeb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:47.618882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:47.644356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf70a559b2444e788e7ab195e6c2aaf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.644356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf70a559b2444e788e7ab195e6c2aaf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.655632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf70a559b2444e788e7ab195e6c2aaf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.661506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1068/2144] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:47 2022] DELETE /identity/v3/users/ccf70a559b2444e788e7ab195e6c2aaf => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:47.662390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:47.665691 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:47.741893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b793-b072-4097-9907-0952e45ffdc1 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:47.746929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:47.747551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.747834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.748140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.748800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:47.749113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:47.750778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:47.763037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1069/2145] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:02:47.767763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:47.779505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8855d-0324-4fca-924b-7d8f8b27c8bf admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:47.784866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:47.785492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.785932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.786725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.787046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:47.789056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:47.831558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4e6fde50f14e9482cc72d36ad1faeb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.840955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4e6fde50f14e9482cc72d36ad1faeb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.841930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4e6fde50f14e9482cc72d36ad1faeb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.842583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4e6fde50f14e9482cc72d36ad1faeb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:47.884601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1070/2146] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:47 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:47.886134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:47.887946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:47.959202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc3c13-4faf-464b-9010-b188f1e6ebae admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:47.964573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:47.965186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:47.965464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:47.965738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:47.966405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:47.968354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:47.978731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4e6fde50f14e9482cc72d36ad1faeb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.002636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1077/2147] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:47 2022] DELETE /identity/v3/users/9c4e6fde50f14e9482cc72d36ad1faeb => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:48.002636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.003638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.085715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cf742-491c-4f08-b4b8-4bd1f770e8a6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.091474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.092174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.092511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.092797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] PATH_INFO: `/v3/projects/3938bc7b824d4a25aedaeb9ca3268ca5` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.094199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3938bc7b824d4a25aedaeb9ca3268ca5)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.096540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.201603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1071/2148] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:47 2022] POST /identity/v3/users => generated 293 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:48.213734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.226979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1782413-7150-4ac9-8f41-8c41cd1a96e5 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.232922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.246227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.246227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.246227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.246227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.246227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.248905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1072/2149] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:48.252550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.285557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71193-c133-4c14-b073-6e249d840418 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.294570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3938bc7b824d4a25aedaeb9ca3268ca5', 'request_id': 'req-c7fcf09b-bddf-4fec-8058-4af89606a6a6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.299515 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3938bc7b824d4a25aedaeb9ca3268ca5', 'request_id': 'req-c7fcf09b-bddf-4fec-8058-4af89606a6a6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.307140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3938bc7b824d4a25aedaeb9ca3268ca5', 'request_id': 'req-c7fcf09b-bddf-4fec-8058-4af89606a6a6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.368508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362b695-d839-4260-b770-447a3a31eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.369121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b695-d839-4260-b770-447a3a31eb8e admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.369401 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b695-d839-4260-b770-447a3a31eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.370020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b695-d839-4260-b770-447a3a31eb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.374953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362b695-d839-4260-b770-447a3a31eb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.374953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362b695-d839-4260-b770-447a3a31eb8e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.383435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1073/2150] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:48.384964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0066ce-a0ed-47ed-a874-e7aedc22cffc None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.384964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0066ce-a0ed-47ed-a874-e7aedc22cffc None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.385234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0066ce-a0ed-47ed-a874-e7aedc22cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.405727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ef0066ce-a0ed-47ed-a874-e7aedc22cffc None None] MFA Rules not processed for user `9c60f6eaccad440abff863fe99b40382`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:48.406979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ef0066ce-a0ed-47ed-a874-e7aedc22cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.453840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] The token cache is being invalidate because project 3938bc7b824d4a25aedaeb9ca3268ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:48.454354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.455001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.456581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1078/2151] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:47 2022] DELETE /identity/v3/projects/3938bc7b824d4a25aedaeb9ca3268ca5 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:48.460303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.460303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.478273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcf09b-bddf-4fec-8058-4af89606a6a6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.483135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1074/2152] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:48.483135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81863e9-7ae5-430d-aacb-1fd81b0f2d54 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.483135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81863e9-7ae5-430d-aacb-1fd81b0f2d54 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.483135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81863e9-7ae5-430d-aacb-1fd81b0f2d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.487610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.488496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.489329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.489620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] PATH_INFO: `/v3/projects/a9e92afcc6744da393978638b5b986a5/users/026b4316e9fb4f78ae1e9c7c3fa6c201/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.505007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e81863e9-7ae5-430d-aacb-1fd81b0f2d54 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:48.506503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e81863e9-7ae5-430d-aacb-1fd81b0f2d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.541722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e92afcc6744da393978638b5b986a5, user_id=026b4316e9fb4f78ae1e9c7c3fa6c201, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.542508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1075/2153] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:48.545017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.551335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecc93d-f13b-405a-847d-ec137170587d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.551621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecc93d-f13b-405a-847d-ec137170587d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.551896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecc93d-f13b-405a-847d-ec137170587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.575820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4aecc93d-f13b-405a-847d-ec137170587d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:48.577332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4aecc93d-f13b-405a-847d-ec137170587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.585527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1076/2154] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:48.597158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecc93d-f13b-405a-847d-ec137170587d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.607380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecc93d-f13b-405a-847d-ec137170587d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.612498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.613106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.613389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.613664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.614364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:48.614664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.616979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.630644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1077/2155] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:02:48.634394 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.645973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabf85e-6b55-4ba6-abd6-40d901ca6083 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.650980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099263f-3497-4501-a48c-29bf9ac651ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.651645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099263f-3497-4501-a48c-29bf9ac651ec admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.651923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099263f-3497-4501-a48c-29bf9ac651ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.652193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099263f-3497-4501-a48c-29bf9ac651ec admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.652832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099263f-3497-4501-a48c-29bf9ac651ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.654782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099263f-3497-4501-a48c-29bf9ac651ec admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.681747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:48.693727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1079/2156] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:48 2022] PUT /identity/v3/projects/a9e92afcc6744da393978638b5b986a5/users/026b4316e9fb4f78ae1e9c7c3fa6c201/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:48.699483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.700894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.744179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1078/2157] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:48 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:48.744179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb781d0b-0887-4019-b100-d00ff31fbec5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.744179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb781d0b-0887-4019-b100-d00ff31fbec5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.744179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb781d0b-0887-4019-b100-d00ff31fbec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.764693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-eb781d0b-0887-4019-b100-d00ff31fbec5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:48.765962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-eb781d0b-0887-4019-b100-d00ff31fbec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.780966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc490d44-8974-4f15-92fc-1207c14178d2 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.834230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1079/2158] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:48.835124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb781d0b-0887-4019-b100-d00ff31fbec5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.846644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb781d0b-0887-4019-b100-d00ff31fbec5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:48.847567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.848491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.848976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.849560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] PATH_INFO: `/v3/projects/2703b25174b14d70a496725e5f00fea9` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.851770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2703b25174b14d70a496725e5f00fea9)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.852508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:48.853161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:48.853439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:48.853716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:48.854306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.854415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:48.856454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:48.952456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1080/2159] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:48 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:48.953502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:48.956905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:48.979381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2703b25174b14d70a496725e5f00fea9', 'request_id': 'req-c0522242-d634-4a16-99cf-f1a8efc47b87'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.986379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2703b25174b14d70a496725e5f00fea9', 'request_id': 'req-c0522242-d634-4a16-99cf-f1a8efc47b87'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:48.993127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2703b25174b14d70a496725e5f00fea9', 'request_id': 'req-c0522242-d634-4a16-99cf-f1a8efc47b87'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.024127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6c272-9b5c-44b0-aa3f-460ddc249a86 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:49.061321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] The token cache is being invalidate because project 2703b25174b14d70a496725e5f00fea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:49.061798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.062930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.068475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1080/2160] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:48 2022] DELETE /identity/v3/projects/2703b25174b14d70a496725e5f00fea9 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:49.068475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:49.070867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:49.084454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:49.089051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:49.089362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:49.089362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:49.089362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:49.089362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:49.157936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522242-d634-4a16-99cf-f1a8efc47b87 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:49.161496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:49.163035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:49.173961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:49.176072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1081/2161] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:49.176197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:49.176197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:49.176197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:49.176197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:49.178410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:49.184460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:49.186525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:49.211844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1081/2162] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:49.215891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5692de1-8885-4f11-b1a6-adf30155eeab service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:49.217232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:49.230187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:49.230950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:49.231527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:49.231866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] PATH_INFO: `/v3/users/568a02129f52424c9b54c4829d8136ce` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:49.233852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=568a02129f52424c9b54c4829d8136ce)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:49.236055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:49.258288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11048dd0-b630-4379-8dfc-ae63f3a7defe admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:49.274757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:49.278667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:49.278667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:49.278667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:49.278667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:49.278667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:49.318793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1082/2163] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:49.319816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:49.338412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad25c3-ce40-46f1-b3ac-ad9cbdd35e22 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:49.369044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881c424-aad9-43fc-8e69-ebce789a521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:49.369866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881c424-aad9-43fc-8e69-ebce789a521f admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:49.370148 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881c424-aad9-43fc-8e69-ebce789a521f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:49.370427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881c424-aad9-43fc-8e69-ebce789a521f admin admin] PATH_INFO: `/v3/projects/302374e5aa464db0997b7618ec2e0f7c/users/d2658d34e6fc4f8991fd33fb6ebc1011/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:49.393652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881c424-aad9-43fc-8e69-ebce789a521f admin admin] RBAC: Authorizing `identity:create_grant(project_id=302374e5aa464db0997b7618ec2e0f7c, user_id=d2658d34e6fc4f8991fd33fb6ebc1011, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:49.396759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881c424-aad9-43fc-8e69-ebce789a521f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:49.627913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1083/2164] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:49 2022] PUT /identity/v3/projects/302374e5aa464db0997b7618ec2e0f7c/users/d2658d34e6fc4f8991fd33fb6ebc1011/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:49.639769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:49.640175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:49.640560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:49.644361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568a02129f52424c9b54c4829d8136ce'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.651983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568a02129f52424c9b54c4829d8136ce'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.669808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] MFA Rules not processed for user `d2658d34e6fc4f8991fd33fb6ebc1011`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:49.671050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:49.739364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1084/2165] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:49.751587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:49.752955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:49.756364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568a02129f52424c9b54c4829d8136ce'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.757680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568a02129f52424c9b54c4829d8136ce'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.775673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568a02129f52424c9b54c4829d8136ce'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:49.782227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1082/2166] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:49 2022] DELETE /identity/v3/users/568a02129f52424c9b54c4829d8136ce => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:49.829973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6ddde-b8ff-491e-9666-423e441b1836 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:49.881001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:49.881757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:49.882626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:49.882901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:49.884657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:49.948149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:49.949610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:49.958558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1085/2167] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:49.985245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:49.985245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:50.087083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580aeb6-fd56-4c20-b2fb-12301e3385c8 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:50.092711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:50.093574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.094280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.096345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] PATH_INFO: `/v3/users/d2658d34e6fc4f8991fd33fb6ebc1011` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.097829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2658d34e6fc4f8991fd33fb6ebc1011)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:50.099976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:50.346817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:50.349510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:50.387651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2658d34e6fc4f8991fd33fb6ebc1011'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.399945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be27a59-03eb-4d1e-9345-7b3f223197e4 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:50.415108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:50.415737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2658d34e6fc4f8991fd33fb6ebc1011'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.416305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.416811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.417098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] PATH_INFO: `/v3/projects/e7afae0aec2c4243a04f9e12c5b9b6b6` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.418483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7afae0aec2c4243a04f9e12c5b9b6b6)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:50.420414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:50.481728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:50.538739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2658d34e6fc4f8991fd33fb6ebc1011'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.539362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2658d34e6fc4f8991fd33fb6ebc1011'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.555983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d0f349e9-0489-43f3-9eb4-ca05f2385657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2658d34e6fc4f8991fd33fb6ebc1011'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.563231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1083/2168] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:49 2022] DELETE /identity/v3/users/d2658d34e6fc4f8991fd33fb6ebc1011 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:50.570012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.570573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.579484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.591421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7afae0aec2c4243a04f9e12c5b9b6b6', 'request_id': 'req-e6d37957-9a99-491b-9cb5-00366334af3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.594627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:50.595878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:50.598757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7afae0aec2c4243a04f9e12c5b9b6b6', 'request_id': 'req-e6d37957-9a99-491b-9cb5-00366334af3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.605712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7afae0aec2c4243a04f9e12c5b9b6b6', 'request_id': 'req-e6d37957-9a99-491b-9cb5-00366334af3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.675491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1084/2169] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:50.685839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] The token cache is being invalidate because project e7afae0aec2c4243a04f9e12c5b9b6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:50.686412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.686531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:50.687369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e6d37957-9a99-491b-9cb5-00366334af3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:50.689124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:50.689258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1086/2170] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:50 2022] DELETE /identity/v3/projects/e7afae0aec2c4243a04f9e12c5b9b6b6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:50.693910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb89c82-db1f-493c-9279-6d68450cb643 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.694374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb89c82-db1f-493c-9279-6d68450cb643 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.694745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb89c82-db1f-493c-9279-6d68450cb643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.716655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-fdb89c82-db1f-493c-9279-6d68450cb643 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:50.718051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fdb89c82-db1f-493c-9279-6d68450cb643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:50.728213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1087/2171] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:50.739145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0961-e1ff-4cf8-adac-b359221c2dd6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.739861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0961-e1ff-4cf8-adac-b359221c2dd6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.740475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0961-e1ff-4cf8-adac-b359221c2dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.767615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-8b1c0961-e1ff-4cf8-adac-b359221c2dd6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:50.769402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8b1c0961-e1ff-4cf8-adac-b359221c2dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:50.779762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1088/2172] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:50.784539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0961-e1ff-4cf8-adac-b359221c2dd6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:50.794441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0961-e1ff-4cf8-adac-b359221c2dd6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:50.800108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:50.800970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.801335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.801711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.804885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:50.807536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:50.807655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94d7fe-f209-4230-aae4-9ae02bba828d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:50.808103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:50.810622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:50.811412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.811703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.811979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.813740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:50.829223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:50.831022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:50.832040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1089/2173] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:02:50.836831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:50.843303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1085/2174] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:50.860064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ced1f-d144-4427-a595-d2c4c891624d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:50.865168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:50.865885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:50.866197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:50.866476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:50.867120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:50.869059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:50.991777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1090/2175] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:50 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:50.995743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:51.008761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680435ad-ceed-4cb0-9f5b-fb41d83a4319 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:51.104136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:51.104874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.105655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.105655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.106407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:51.108754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:51.258492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1086/2176] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:50 2022] POST /identity/v3/users => generated 292 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:51.259234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:51.270692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1770dd2-c929-48dc-8463-3ff3da5800d1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:51.281495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:51.282180 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.282465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.282733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.283384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:51.285020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:51.297356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1091/2177] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:51.301541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:51.312433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f7742-0fe9-42e4-8408-bed4f1339b53 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:51.318575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:51.319242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.319780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.320069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.321031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:51.323083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:51.333672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1087/2178] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:51.345438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:51.348683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fdbad-6eda-4311-9945-4d8aa1e590ae admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:51.353951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b5f7e-2e3f-4962-a2da-d1fd861966dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:51.354625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5f7e-2e3f-4962-a2da-d1fd861966dd admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.354909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5f7e-2e3f-4962-a2da-d1fd861966dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.355201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5f7e-2e3f-4962-a2da-d1fd861966dd admin admin] PATH_INFO: `/v3/projects/51ead17283af4836a91bfd77c197ec59/users/4c0af25c332b490db1e2b9be67027e8f/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.378225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:51.381241 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:51.396691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5f7e-2e3f-4962-a2da-d1fd861966dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ead17283af4836a91bfd77c197ec59, user_id=4c0af25c332b490db1e2b9be67027e8f, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:51.399914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5f7e-2e3f-4962-a2da-d1fd861966dd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:51.404281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13866a1-6bbe-44cd-b4fc-c5997e725804 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:51.421092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:51.421747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.422044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.422316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] PATH_INFO: `/v3/projects/302374e5aa464db0997b7618ec2e0f7c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.426869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=302374e5aa464db0997b7618ec2e0f7c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:51.429069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:51.500143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1092/2179] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:51 2022] PUT /identity/v3/projects/51ead17283af4836a91bfd77c197ec59/users/4c0af25c332b490db1e2b9be67027e8f/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:51.511640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12b661-33b9-471d-af1b-db1ce77a7a25 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.512002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12b661-33b9-471d-af1b-db1ce77a7a25 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.512344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12b661-33b9-471d-af1b-db1ce77a7a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.561905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-df12b661-33b9-471d-af1b-db1ce77a7a25 None None] MFA Rules not processed for user `4c0af25c332b490db1e2b9be67027e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:51.563166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-df12b661-33b9-471d-af1b-db1ce77a7a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:51.608381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302374e5aa464db0997b7618ec2e0f7c', 'request_id': 'req-da274323-4e38-4a7a-88cf-4f6b0014b8ef'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:51.615736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302374e5aa464db0997b7618ec2e0f7c', 'request_id': 'req-da274323-4e38-4a7a-88cf-4f6b0014b8ef'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:51.663559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1093/2180] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:51.668173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12b661-33b9-471d-af1b-db1ce77a7a25 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:51.719759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302374e5aa464db0997b7618ec2e0f7c', 'request_id': 'req-da274323-4e38-4a7a-88cf-4f6b0014b8ef'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:51.744246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] The token cache is being invalidate because project 302374e5aa464db0997b7618ec2e0f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:51.744608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:51.746866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-da274323-4e38-4a7a-88cf-4f6b0014b8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:51.753828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1088/2181] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:51 2022] DELETE /identity/v3/projects/302374e5aa464db0997b7618ec2e0f7c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:51.757336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39216c-e97b-41a1-9bd8-bb98ebe65b17 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.762416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39216c-e97b-41a1-9bd8-bb98ebe65b17 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.762416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39216c-e97b-41a1-9bd8-bb98ebe65b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.781757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-bc39216c-e97b-41a1-9bd8-bb98ebe65b17 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:51.783061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bc39216c-e97b-41a1-9bd8-bb98ebe65b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:51.790448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12b661-33b9-471d-af1b-db1ce77a7a25 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:51.887445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:51.888264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.888852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.889224 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.890245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:51.892319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:51.901364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1089/2182] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:51.910943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ab9f-4a86-42cf-890a-c9477fb1e9ea None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.911298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ab9f-4a86-42cf-890a-c9477fb1e9ea None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.911645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ab9f-4a86-42cf-890a-c9477fb1e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.952247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-6ab4ab9f-4a86-42cf-890a-c9477fb1e9ea None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:51.953615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6ab4ab9f-4a86-42cf-890a-c9477fb1e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:51.967123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1090/2183] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:51.969385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4ab9f-4a86-42cf-890a-c9477fb1e9ea None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:51.980654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4ab9f-4a86-42cf-890a-c9477fb1e9ea None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:51.985537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:51.987177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:51.987470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:51.987841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:51.988262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:51.988564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:51.990219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.004114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1091/2184] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:02:52.009134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.022833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06f737-2675-420a-a551-1e29eb8346f9 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.028932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1094/2185] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:51 2022] POST /identity/v3/projects => generated 383 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:52.029606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.030271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.031064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.031351 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.032001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.033371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.034750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.035941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:52.055294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:52.127825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c8da9-d146-4416-b0b9-98b2a0f85627 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.140703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1092/2186] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:52 2022] POST /identity/v3/projects => generated 361 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:52.141319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.146414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.147045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.147354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.147834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.149115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.151568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.279981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1095/2187] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:52 2022] POST /identity/v3/users => generated 292 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:52.283934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.286034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5c3f-34fb-4148-b786-e5aba816a426 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.292204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.292946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.293332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.293727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.294594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.296983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.301676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b6e7f-f9bd-4d45-a350-b75f101257cc admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.308927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.310198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.310928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.311352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.312635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.314494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.326375 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1096/2188] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:02:52.335286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.363405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a60fb-3128-4c4c-9e5d-2ba6fe08d177 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.364052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.364663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.364913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.365240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.366172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.368453 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.383039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1097/2189] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:52.420047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.442753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065e346-9723-4ffe-b3e0-91ae70d0200d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.445383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1093/2190] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:52 2022] POST /identity/v3/users => generated 281 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:52.449427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.456767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea8e6-e2c5-49a3-be2c-706dd36bb763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.457480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea8e6-e2c5-49a3-be2c-706dd36bb763 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.457762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea8e6-e2c5-49a3-be2c-706dd36bb763 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.466551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b58d5-eed4-47f5-838a-73a5c7dc6b51 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.466991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea8e6-e2c5-49a3-be2c-706dd36bb763 admin admin] PATH_INFO: `/v3/projects/d41210beed51442c9f6949eb28a9a128/users/52cc02984f81485fb17d9d611e488446/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.480878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.482183 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.482569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.482973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.483885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.486004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.501681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1094/2191] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:52.505114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.505273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea8e6-e2c5-49a3-be2c-706dd36bb763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41210beed51442c9f6949eb28a9a128, user_id=52cc02984f81485fb17d9d611e488446, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.508765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea8e6-e2c5-49a3-be2c-706dd36bb763 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.517698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d394e-8618-4273-be0d-4282e9489bdd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.523599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.524836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.525365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.526180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.527234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.530308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.548588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1095/2192] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:52.549926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.564230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3600-be2f-4742-8540-21f854ea6dba admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:52.570524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3ea0e-8f1b-473a-8600-38a676f42579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:52.571382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3ea0e-8f1b-473a-8600-38a676f42579 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.571735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3ea0e-8f1b-473a-8600-38a676f42579 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.572081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3ea0e-8f1b-473a-8600-38a676f42579 admin admin] PATH_INFO: `/v3/projects/9111d22a47c84272a252e9cc58d78627/users/e63e2bbd7bec43f7b0d37e9859e7c279/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.599088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3ea0e-8f1b-473a-8600-38a676f42579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9111d22a47c84272a252e9cc58d78627, user_id=e63e2bbd7bec43f7b0d37e9859e7c279, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:52.601340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3ea0e-8f1b-473a-8600-38a676f42579 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:52.629514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1098/2193] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:52 2022] PUT /identity/v3/projects/d41210beed51442c9f6949eb28a9a128/users/52cc02984f81485fb17d9d611e488446/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:52.653262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8dfb0-d3f3-4e54-b63a-8b4dcffd3752 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.653572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8dfb0-d3f3-4e54-b63a-8b4dcffd3752 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.653903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8dfb0-d3f3-4e54-b63a-8b4dcffd3752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.684988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-50f8dfb0-d3f3-4e54-b63a-8b4dcffd3752 None None] MFA Rules not processed for user `52cc02984f81485fb17d9d611e488446`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:52.686510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-50f8dfb0-d3f3-4e54-b63a-8b4dcffd3752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:52.700645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1096/2194] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:52 2022] PUT /identity/v3/projects/9111d22a47c84272a252e9cc58d78627/users/e63e2bbd7bec43f7b0d37e9859e7c279/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:52.711476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:52.711824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:52.712118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:52.759636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] MFA Rules not processed for user `e63e2bbd7bec43f7b0d37e9859e7c279`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:52.760938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:52.824472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1099/2195] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:52.834582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8dfb0-d3f3-4e54-b63a-8b4dcffd3752 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.874204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1097/2196] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:52.929487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:52.929487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:52.946130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8dfb0-d3f3-4e54-b63a-8b4dcffd3752 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.008476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5a478-7507-4b1c-a860-eede1ba3699e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.012760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.013390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.013725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.014030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.027046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.029941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.036183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.037193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.038204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.038492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.039204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.041359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.062579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1098/2197] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:53.066970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:53.093296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a03e-ed2a-4441-8214-a3c35be54495 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.099495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.100491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.100947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.101425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] PATH_INFO: `/v3/users/e63e2bbd7bec43f7b0d37e9859e7c279` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.103445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63e2bbd7bec43f7b0d37e9859e7c279)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.106512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.172022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1100/2198] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:52 2022] POST /identity/v3/projects => generated 383 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:53.177104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:53.190501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:53.256581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef426c8-0b8d-4d11-90c0-cb862a5b20c6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.262972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.263681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.263958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.264233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.264888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.266898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.392034 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1101/2199] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:53 2022] POST /identity/v3/users => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:53.397763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:53.414258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63e2bbd7bec43f7b0d37e9859e7c279'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:53.420162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1630817-8eb9-4d1f-bd4f-d11a37f6ac32 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.431836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63e2bbd7bec43f7b0d37e9859e7c279'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:53.432623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.435165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.436378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.436689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.437685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.441802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.462633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1102/2200] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:53.463697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:53.477045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28633463-f622-4ce7-990f-eee8e0bf47ad admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.483048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.483735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.484154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.484441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] PATH_INFO: `/v3/projects/3072f8a6c1da4ab7a7dab05dd90cf34a/users/fd639d2a1b1f47c3833835b039d5382c/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.506062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3072f8a6c1da4ab7a7dab05dd90cf34a, user_id=fd639d2a1b1f47c3833835b039d5382c, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.511452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.539174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63e2bbd7bec43f7b0d37e9859e7c279'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:53.540090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63e2bbd7bec43f7b0d37e9859e7c279'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:53.561580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-523b3e9f-a26a-4385-a420-b0eaa3137526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63e2bbd7bec43f7b0d37e9859e7c279'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:53.573847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1099/2201] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:53 2022] DELETE /identity/v3/users/e63e2bbd7bec43f7b0d37e9859e7c279 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:53.580019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.580547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.580854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.604608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:53.606733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:53.658415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1103/2202] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:53 2022] PUT /identity/v3/projects/3072f8a6c1da4ab7a7dab05dd90cf34a/users/fd639d2a1b1f47c3833835b039d5382c/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:53.663565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:53.667004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:53.744281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1100/2203] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:53.769334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:53.770675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:53.838282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fc860-e803-433b-8f7e-b075aba8b8a1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.854753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.855609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.855985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.856591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.858135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.862380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.862540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4e0c2-b746-42be-a0da-dd02b2043307 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.880754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1104/2204] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:53.883596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:53.900571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71aa3-828d-46a7-8dc7-ee35a0fd7a78 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:53.907207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf1011-0505-4888-9279-f8fe934ac0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.908558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf1011-0505-4888-9279-f8fe934ac0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.909060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf1011-0505-4888-9279-f8fe934ac0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.909436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf1011-0505-4888-9279-f8fe934ac0f8 admin admin] PATH_INFO: `/v3/projects/3072f8a6c1da4ab7a7dab05dd90cf34a/users/fd639d2a1b1f47c3833835b039d5382c/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.911754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf1011-0505-4888-9279-f8fe934ac0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3072f8a6c1da4ab7a7dab05dd90cf34a, user_id=fd639d2a1b1f47c3833835b039d5382c, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.919238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf1011-0505-4888-9279-f8fe934ac0f8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:53.943889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1105/2205] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:53 2022] PUT /identity/v3/projects/3072f8a6c1da4ab7a7dab05dd90cf34a/users/fd639d2a1b1f47c3833835b039d5382c/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:53.954467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.954841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.955178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.980961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:53.982068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:53.982484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:53.983349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:53.988762 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] MFA Rules not processed for user `fd639d2a1b1f47c3833835b039d5382c`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:53.991088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:53.996939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:53.999308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:54.019898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1101/2206] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:54.123618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1106/2207] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:54.136997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:54.138724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:54.227440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3220fc-2749-4d57-987c-3e94c8318741 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:54.300432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32041798-ee87-4866-b87b-2cb820fd0911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:54.301237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32041798-ee87-4866-b87b-2cb820fd0911 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:54.301463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32041798-ee87-4866-b87b-2cb820fd0911 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:54.301849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32041798-ee87-4866-b87b-2cb820fd0911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:54.304487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-32041798-ee87-4866-b87b-2cb820fd0911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:54.405878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32041798-ee87-4866-b87b-2cb820fd0911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:54.408764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32041798-ee87-4866-b87b-2cb820fd0911 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:54.431141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1102/2208] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:54.567703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:54.820879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:54.823371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:54.927896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b61649-dad7-4c62-9021-39421a55d948 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:54.933860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:54.934555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:54.934847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:54.935119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] PATH_INFO: `/v3/projects/9111d22a47c84272a252e9cc58d78627` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:54.936909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9111d22a47c84272a252e9cc58d78627)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:54.938767 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:55.144344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9111d22a47c84272a252e9cc58d78627', 'request_id': 'req-1e1ded31-7938-41ed-9bb1-e7b069f9b892'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:55.166485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9111d22a47c84272a252e9cc58d78627', 'request_id': 'req-1e1ded31-7938-41ed-9bb1-e7b069f9b892'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:55.177525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9111d22a47c84272a252e9cc58d78627', 'request_id': 'req-1e1ded31-7938-41ed-9bb1-e7b069f9b892'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:55.338369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] The token cache is being invalidate because project 9111d22a47c84272a252e9cc58d78627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:55.338851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:55.339945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e1ded31-7938-41ed-9bb1-e7b069f9b892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:55.344311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1107/2209] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:54 2022] DELETE /identity/v3/projects/9111d22a47c84272a252e9cc58d78627 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:55.349496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498f41-8712-4a79-983d-75cfd0c401a9 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:55.350313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498f41-8712-4a79-983d-75cfd0c401a9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:55.350313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498f41-8712-4a79-983d-75cfd0c401a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:55.380388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d8498f41-8712-4a79-983d-75cfd0c401a9 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:55.382451 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d8498f41-8712-4a79-983d-75cfd0c401a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:55.454434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1103/2210] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:55.462717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906965d-ed9a-42a7-87de-9f111e0f2456 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:55.463020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906965d-ed9a-42a7-87de-9f111e0f2456 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:55.463290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906965d-ed9a-42a7-87de-9f111e0f2456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:55.485494 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c906965d-ed9a-42a7-87de-9f111e0f2456 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:55.486824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c906965d-ed9a-42a7-87de-9f111e0f2456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:55.499838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1108/2211] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:55.501303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498f41-8712-4a79-983d-75cfd0c401a9 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:55.515243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498f41-8712-4a79-983d-75cfd0c401a9 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:55.521811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:55.523063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:55.523063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:55.523370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:55.524252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:02:55.524693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:55.527672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:55.545904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1104/2212] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:02:55.549084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906965d-ed9a-42a7-87de-9f111e0f2456 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:55.560303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906965d-ed9a-42a7-87de-9f111e0f2456 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:55.573586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:55.574320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:55.574539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:55.574841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:55.575850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:55.577851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:55.605943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:55.677638 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1109/2213] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:55 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:55.685039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:55.696327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24921363-8f15-46c5-bc4f-2f59ff10aeb4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:55.787649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:55.790432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:55.790719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:55.791016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:55.791689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:55.794006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:55.952882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1105/2214] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:55 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:02:55.957379 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:55.968773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099060ce-0cea-44dd-ba0e-479f0a0a1851 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:55.982296 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:55.983340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:55.983917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:55.984405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:55.985970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:55.988252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:55.990721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:56.011093 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1110/2215] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:56.012781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:56.030445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f9b8-6db9-4ba7-a7e8-bc179fc66d08 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:56.039848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90959cdd-90b4-434d-8ec5-44da110f2e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:56.041149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959cdd-90b4-434d-8ec5-44da110f2e4f admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:56.041644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959cdd-90b4-434d-8ec5-44da110f2e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:56.042164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959cdd-90b4-434d-8ec5-44da110f2e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:56.043623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90959cdd-90b4-434d-8ec5-44da110f2e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:56.045733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90959cdd-90b4-434d-8ec5-44da110f2e4f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:56.063713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1106/2216] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:02:56.068168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:56.082901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7687873-5ea3-4229-8a74-d97f1968e9eb admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:56.094753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fce30d-229a-4c73-aea8-729239e4bdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:56.095936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce30d-229a-4c73-aea8-729239e4bdd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:56.096434 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce30d-229a-4c73-aea8-729239e4bdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:56.096948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce30d-229a-4c73-aea8-729239e4bdd5 admin admin] PATH_INFO: `/v3/projects/042844f069544a05976c952578fa2591/users/2401af0b162f4d16b8e92983b4a431f0/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:56.122939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fce30d-229a-4c73-aea8-729239e4bdd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=042844f069544a05976c952578fa2591, user_id=2401af0b162f4d16b8e92983b4a431f0, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:56.126451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fce30d-229a-4c73-aea8-729239e4bdd5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:56.221808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1111/2217] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:02:56 2022] PUT /identity/v3/projects/042844f069544a05976c952578fa2591/users/2401af0b162f4d16b8e92983b4a431f0/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:02:56.239446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:56.240290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:56.240751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:56.298411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] MFA Rules not processed for user `2401af0b162f4d16b8e92983b4a431f0`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:56.301569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:56.391665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec909-9832-42e5-a14f-4536081a5489 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:56.392162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec909-9832-42e5-a14f-4536081a5489 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:56.392441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec909-9832-42e5-a14f-4536081a5489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:56.416315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1107/2218] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:56.432161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:56.433696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:56.436163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-1e1ec909-9832-42e5-a14f-4536081a5489 None None] MFA Rules not processed for user `c0afa15991f14e0ca0d3701d06a89049`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:56.437672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1e1ec909-9832-42e5-a14f-4536081a5489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:56.558542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1112/2219] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:56.558723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f77f22-9a4b-4a07-a61f-aaa81eff0701 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:56.572880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:56.573527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:56.573828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:56.574104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:56.590479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:56.592272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:56.610066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1108/2220] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:56.655686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec909-9832-42e5-a14f-4536081a5489 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:56.685474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:56.698075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ca3b-98df-4e07-964d-ca8b063bc464 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:56.712593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613583d8-742e-4025-97b2-929767663ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:56.713235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613583d8-742e-4025-97b2-929767663ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:56.713608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613583d8-742e-4025-97b2-929767663ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:56.714063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613583d8-742e-4025-97b2-929767663ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:56.721135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-613583d8-742e-4025-97b2-929767663ca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:56.758410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec909-9832-42e5-a14f-4536081a5489 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:56.765993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:56.767340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:56.767655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:56.767935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] PATH_INFO: `/v3/users/2401af0b162f4d16b8e92983b4a431f0` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:56.769332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2401af0b162f4d16b8e92983b4a431f0)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:56.772104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:56.834238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613583d8-742e-4025-97b2-929767663ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:56.836306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613583d8-742e-4025-97b2-929767663ca2 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:56.848086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1109/2221] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:02:57.002390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2401af0b162f4d16b8e92983b4a431f0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:57.010854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2401af0b162f4d16b8e92983b4a431f0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:57.011608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2401af0b162f4d16b8e92983b4a431f0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:57.012213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2401af0b162f4d16b8e92983b4a431f0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:57.012593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:57.126758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2401af0b162f4d16b8e92983b4a431f0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:57.147606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1113/2222] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:02:56 2022] DELETE /identity/v3/users/2401af0b162f4d16b8e92983b4a431f0 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:57.152228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:57.152518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:57.152838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:57.171850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:57.173064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:57.296488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1110/2223] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:02:57.321292 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:57.323363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:57.388725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:57.449468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7193c2b-d0ec-4124-a692-9a73797dc208 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:57.463757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:57.464967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:57.465252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:57.467328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:57.495142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:57.497582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:57.513384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1114/2224] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:58.680651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:58.682247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:58.715807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff4136-35cc-4e93-91d2-400f6bc3bb8b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:58.733115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:58.735966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:58.739292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:58.739292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] PATH_INFO: `/v3/projects/042844f069544a05976c952578fa2591` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:58.741667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042844f069544a05976c952578fa2591)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:58.747641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:58.895577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042844f069544a05976c952578fa2591', 'request_id': 'req-ef023229-4322-4066-8d90-7d43232e4ce3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:58.901403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042844f069544a05976c952578fa2591', 'request_id': 'req-ef023229-4322-4066-8d90-7d43232e4ce3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:58.972606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042844f069544a05976c952578fa2591', 'request_id': 'req-ef023229-4322-4066-8d90-7d43232e4ce3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:58.990719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] The token cache is being invalidate because project 042844f069544a05976c952578fa2591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:02:58.991564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:58.994210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ef023229-4322-4066-8d90-7d43232e4ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:02:59.017803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1111/2225] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:02:58 2022] DELETE /identity/v3/projects/042844f069544a05976c952578fa2591 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:02:59.029830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:59.032921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:59.098585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:59.144835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216153c-701b-4bc2-bc26-d1dcf8d39596 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:59.243331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:02:59.307881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:59.308517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:59.308793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:59.309282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:59.312758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:59.409119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:59.411060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:59.422280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1115/2226] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:02:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:02:59.513618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7ea01-3c80-426d-a792-561f2b1d1cd1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:59.513947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7ea01-3c80-426d-a792-561f2b1d1cd1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:59.514482 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7ea01-3c80-426d-a792-561f2b1d1cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:59.544935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0aa7ea01-3c80-426d-a792-561f2b1d1cd1 None None] MFA Rules not processed for user `026b4316e9fb4f78ae1e9c7c3fa6c201`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:02:59.546396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0aa7ea01-3c80-426d-a792-561f2b1d1cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:59.658928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1112/2227] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:02:59.660700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:59.662474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:02:59.810894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3521fca-2fa3-441b-9301-1237e28ac055 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:02:59.815590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:02:59.816197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:02:59.816473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:02:59.816804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:02:59.817382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:02:59.819866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:02:59.961410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1116/2228] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:02:59 2022] POST /identity/v3/projects => generated 383 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:02:59.961750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7ea01-3c80-426d-a792-561f2b1d1cd1 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:02:59.973144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7ea01-3c80-426d-a792-561f2b1d1cd1 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:00.033803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:00.034453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:00.034759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:00.035042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:00.035710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:00.037675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:00.122771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1113/2229] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:02:59 2022] POST /identity/v3/users => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:03:00.128041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:00.140439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc0d08-3593-4b2d-9cde-efe2ebfea664 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:00.145993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:00.146614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:00.146897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:00.147290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:00.148006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:00.149721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:00.161615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1117/2230] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:03:00.170844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:00.184362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fa55b-e9ab-471c-8996-bd28c1add3e6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:00.191801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:00.196104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:00.196104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:00.196104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] PATH_INFO: `/v3/projects/f7b4da64338a4468b1b7d0ac77c61e47/users/248a831de51e4d518f22ad7531e32c93/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:00.209479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b4da64338a4468b1b7d0ac77c61e47, user_id=248a831de51e4d518f22ad7531e32c93, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:00.212276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:00.321494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1114/2231] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:03:00 2022] PUT /identity/v3/projects/f7b4da64338a4468b1b7d0ac77c61e47/users/248a831de51e4d518f22ad7531e32c93/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:03:00.325542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:00.336769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7141c-facd-4eb8-a272-fb45b4daf67c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:00.404777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae943c25-222c-4f89-a841-f56ccb9419e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:00.405589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943c25-222c-4f89-a841-f56ccb9419e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:00.405941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943c25-222c-4f89-a841-f56ccb9419e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:00.406221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943c25-222c-4f89-a841-f56ccb9419e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:00.408135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae943c25-222c-4f89-a841-f56ccb9419e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:00.409371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae943c25-222c-4f89-a841-f56ccb9419e1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:00.425811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:00.430231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1118/2232] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:03:00.438519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e86c69-be27-4d52-b513-cdfd3c55b452 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:00.453517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:00.458543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:00.459336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:00.459671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:00.459943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] PATH_INFO: `/v3/projects/f7b4da64338a4468b1b7d0ac77c61e47/users/248a831de51e4d518f22ad7531e32c93/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:00.462386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b4da64338a4468b1b7d0ac77c61e47, user_id=248a831de51e4d518f22ad7531e32c93, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:00.466682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:00.479164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1115/2233] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:03:00 2022] PUT /identity/v3/projects/f7b4da64338a4468b1b7d0ac77c61e47/users/248a831de51e4d518f22ad7531e32c93/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:03:00.938739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:02.369813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:02.376529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:02.376529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:02.411647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:02.414468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:02.533038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1119/2234] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:03:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:02.589908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:02.601663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:02.832080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9793dde-e16c-4d0d-a8da-cb451566adf6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:02.839236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521a48d-df4b-494e-ac56-e343a6920d7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:02.840380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a48d-df4b-494e-ac56-e343a6920d7f service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:02.840875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a48d-df4b-494e-ac56-e343a6920d7f service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:02.841360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a48d-df4b-494e-ac56-e343a6920d7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:02.853221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521a48d-df4b-494e-ac56-e343a6920d7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:02.856350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521a48d-df4b-494e-ac56-e343a6920d7f service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:02.888695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1116/2235] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:03:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:03:03.042012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:03.818764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:04.567448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:04.916912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:04.923774 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:04.961287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:05.170324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da89e96-1b2a-4d7f-a775-940dd2f6600b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:05.189353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:05.190101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:05.190508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:05.190629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] PATH_INFO: `/v3/users/4c0af25c332b490db1e2b9be67027e8f` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:05.191828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0af25c332b490db1e2b9be67027e8f)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:05.194174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:05.506024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:05.520758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0af25c332b490db1e2b9be67027e8f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:05.540715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0af25c332b490db1e2b9be67027e8f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:05.550419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0af25c332b490db1e2b9be67027e8f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:05.553455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0af25c332b490db1e2b9be67027e8f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:06.007046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0af25c332b490db1e2b9be67027e8f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:06.061346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1120/2236] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:03:04 2022] DELETE /identity/v3/users/4c0af25c332b490db1e2b9be67027e8f => generated 0 bytes in 1142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:03:06.063488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:06.064346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:06.064785 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:06.095899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:06.097966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:06.257048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1117/2237] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:06.276301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:06.278978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:06.430898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9124b-8801-4302-ad9f-a3126cb2def9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:06.442821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:06.444301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:06.444609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:06.444876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:06.482889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:06.495729 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:06.536418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1121/2238] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:06.853695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:06.893229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:07.407720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:08.207752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:08.207752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:08.220230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501d610-dffa-4210-b983-e69642dc84ac None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:08.228552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:08.229508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:08.242055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:08.242055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] PATH_INFO: `/v3/projects/51ead17283af4836a91bfd77c197ec59` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:08.242055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ead17283af4836a91bfd77c197ec59)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:08.242055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:08.451852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ead17283af4836a91bfd77c197ec59', 'request_id': 'req-2af2bc11-9842-462d-8069-4f1f1db4cbdf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:08.475829 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ead17283af4836a91bfd77c197ec59', 'request_id': 'req-2af2bc11-9842-462d-8069-4f1f1db4cbdf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:08.595558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ead17283af4836a91bfd77c197ec59', 'request_id': 'req-2af2bc11-9842-462d-8069-4f1f1db4cbdf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:08.619090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] The token cache is being invalidate because project 51ead17283af4836a91bfd77c197ec59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:03:08.619403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:08.620095 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:08.623234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1118/2239] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:03:08 2022] DELETE /identity/v3/projects/51ead17283af4836a91bfd77c197ec59 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:03:08.626875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:08.656284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452c3e6-6dcd-46c9-af3c-963ce6874e80 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:08.846550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:08.847653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:08.848125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:08.848566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] PATH_INFO: `/v3/users/52cc02984f81485fb17d9d611e488446` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:08.853431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=52cc02984f81485fb17d9d611e488446)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:08.858669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:09.120172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:09.133643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52cc02984f81485fb17d9d611e488446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:09.151573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52cc02984f81485fb17d9d611e488446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:09.153124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52cc02984f81485fb17d9d611e488446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:09.153858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52cc02984f81485fb17d9d611e488446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:09.258891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8e75faec-472a-4b9c-bdae-54015ca17d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52cc02984f81485fb17d9d611e488446'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:09.284254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:09.324163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1122/2240] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:03:08 2022] DELETE /identity/v3/users/52cc02984f81485fb17d9d611e488446 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:03:10.048355 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:11.336478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:11.347672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:11.381863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2bc11-9842-462d-8069-4f1f1db4cbdf admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:11.400882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:11.402497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:11.402970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:11.440199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] MFA Rules not processed for user `248a831de51e4d518f22ad7531e32c93`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:11.442003 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:11.542068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1123/2241] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:03:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:11.558987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:11.561260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:11.572550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:11.574052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:11.574611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:11.575036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] PATH_INFO: `/v3/projects/d41210beed51442c9f6949eb28a9a128` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:11.586334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41210beed51442c9f6949eb28a9a128)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:11.588981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:11.663109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9540672-586c-4323-983f-3c07eba64c70 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:11.663289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:11.664657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:11.664952 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:11.665231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:11.667506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:11.773701 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41210beed51442c9f6949eb28a9a128', 'request_id': 'req-a675b18f-754a-4629-b33f-c9a5ca1034fa'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:11.780957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:11.794612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41210beed51442c9f6949eb28a9a128', 'request_id': 'req-a675b18f-754a-4629-b33f-c9a5ca1034fa'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:11.797291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:11.827732 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1124/2242] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:03:11.919228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41210beed51442c9f6949eb28a9a128', 'request_id': 'req-a675b18f-754a-4629-b33f-c9a5ca1034fa'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:11.933677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] The token cache is being invalidate because project d41210beed51442c9f6949eb28a9a128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:03:11.934175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:11.934890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a675b18f-754a-4629-b33f-c9a5ca1034fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:11.937213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1119/2243] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:03:11 2022] DELETE /identity/v3/projects/d41210beed51442c9f6949eb28a9a128 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:03:11.940911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:11.943263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:11.971095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cac4d-e248-4e6a-9bd7-2b1f1ae3f295 service glance] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:12.159380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:12.160089 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:12.160432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:12.160712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] PATH_INFO: `/v3/users/fd639d2a1b1f47c3833835b039d5382c` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:12.162650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd639d2a1b1f47c3833835b039d5382c)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:12.166058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:12.327133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:12.327741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:12.328257 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:12.375696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:12.378057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:12.497188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1120/2244] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:03:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 01 14:03:12.500349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd639d2a1b1f47c3833835b039d5382c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:12.510784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd639d2a1b1f47c3833835b039d5382c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:12.511534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd639d2a1b1f47c3833835b039d5382c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:12.512447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd639d2a1b1f47c3833835b039d5382c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:12.526001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:12.528044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:12.576580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:12.595464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd639d2a1b1f47c3833835b039d5382c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:12.639537 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1125/2245] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:03:11 2022] DELETE /identity/v3/users/fd639d2a1b1f47c3833835b039d5382c => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:03:12.669116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3821e55-e36a-41de-bab0-5b729989ce19 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:12.888365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a1795-b703-4d0f-b42e-09645dcc4993 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:12.889437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1795-b703-4d0f-b42e-09645dcc4993 service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:12.889735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1795-b703-4d0f-b42e-09645dcc4993 service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:12.890038 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1795-b703-4d0f-b42e-09645dcc4993 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:12.898433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-568a1795-b703-4d0f-b42e-09645dcc4993 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:13.043959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a1795-b703-4d0f-b42e-09645dcc4993 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:13.046123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a1795-b703-4d0f-b42e-09645dcc4993 service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:13.056483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1121/2246] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:03:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:03:13.539194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:14.021082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:14.024666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:14.237168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b92e0-6135-4c3d-b5d6-2389a25b76de admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:14.342488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:14.342488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:14.342488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:14.342488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] PATH_INFO: `/v3/projects/3072f8a6c1da4ab7a7dab05dd90cf34a` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:14.343827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3072f8a6c1da4ab7a7dab05dd90cf34a)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:14.343827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:15.050022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3072f8a6c1da4ab7a7dab05dd90cf34a', 'request_id': 'req-1e12aa1b-1cde-4892-97eb-84b0d09e5749'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:15.078215 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3072f8a6c1da4ab7a7dab05dd90cf34a', 'request_id': 'req-1e12aa1b-1cde-4892-97eb-84b0d09e5749'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:15.101939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3072f8a6c1da4ab7a7dab05dd90cf34a', 'request_id': 'req-1e12aa1b-1cde-4892-97eb-84b0d09e5749'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:15.223826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] The token cache is being invalidate because project 3072f8a6c1da4ab7a7dab05dd90cf34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:03:15.223826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:15.223826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-1e12aa1b-1cde-4892-97eb-84b0d09e5749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:03:15.225032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1126/2247] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:03:14 2022] DELETE /identity/v3/projects/3072f8a6c1da4ab7a7dab05dd90cf34a => generated 0 bytes in 1205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:03:15.232146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14229c3d-f5b2-424b-a59c-304e15d13e47 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:15.232743 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14229c3d-f5b2-424b-a59c-304e15d13e47 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:15.233269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14229c3d-f5b2-424b-a59c-304e15d13e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:15.299724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-14229c3d-f5b2-424b-a59c-304e15d13e47 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:15.301349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-14229c3d-f5b2-424b-a59c-304e15d13e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:15.467635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1122/2248] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:15.476223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeaeae-370b-40fd-b804-81180dd16a71 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:15.476784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeaeae-370b-40fd-b804-81180dd16a71 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:15.477324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeaeae-370b-40fd-b804-81180dd16a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:15.521440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-69aeaeae-370b-40fd-b804-81180dd16a71 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:15.524895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-69aeaeae-370b-40fd-b804-81180dd16a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:15.551556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1127/2249] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:15.555916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14229c3d-f5b2-424b-a59c-304e15d13e47 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:15.572387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14229c3d-f5b2-424b-a59c-304e15d13e47 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:15.590690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:15.591479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:15.591850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:15.592227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:15.592988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:03:15.593391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:15.595176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:15.654600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1123/2250] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:03:15.661035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aeaeae-370b-40fd-b804-81180dd16a71 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:15.668673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aeaeae-370b-40fd-b804-81180dd16a71 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:15.678601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:15.679311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:15.679590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:15.679863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:15.680798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:15.685113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:15.704371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:15.859438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1128/2251] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:03:15 2022] POST /identity/v3/projects => generated 367 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:03:15.860617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:15.892932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c11b-f7b7-4d52-85fc-a09aa43e826c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:15.971281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:15.971904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:15.972319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:15.973607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:15.974378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:15.980084 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:16.192090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:16.192639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1124/2252] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:03:15 2022] POST /identity/v3/users => generated 284 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:03:16.209112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dcc2a-fecc-4e2e-a332-d6d11dbc6665 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:16.218254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:16.218975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:16.219275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:16.219552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:16.220299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:16.234066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:16.258498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1129/2253] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:03:16.262441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:16.290050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d57c18-14f5-439d-bb88-0c90d5cedd01 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:16.299710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b627e1-c698-4aaa-9f86-f0a4b453a15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:16.300969 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b627e1-c698-4aaa-9f86-f0a4b453a15b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:16.301510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b627e1-c698-4aaa-9f86-f0a4b453a15b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:16.302100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b627e1-c698-4aaa-9f86-f0a4b453a15b admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:16.303905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b627e1-c698-4aaa-9f86-f0a4b453a15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:16.307353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b627e1-c698-4aaa-9f86-f0a4b453a15b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:16.336053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1125/2254] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:03:16.340896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:16.405551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d117e-bc3d-48cd-83a4-d2ee7fe1dc7a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:16.429933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6ae51-16be-4675-abab-640f0f0f47b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:16.430972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6ae51-16be-4675-abab-640f0f0f47b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:16.431437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6ae51-16be-4675-abab-640f0f0f47b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:16.431867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6ae51-16be-4675-abab-640f0f0f47b2 admin admin] PATH_INFO: `/v3/projects/9d1b07435aac450ca6454eb3f9b09686/users/76a57e0a9c954b52afc6f1979c8d02e0/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:16.480775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6ae51-16be-4675-abab-640f0f0f47b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d1b07435aac450ca6454eb3f9b09686, user_id=76a57e0a9c954b52afc6f1979c8d02e0, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:16.483854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6ae51-16be-4675-abab-640f0f0f47b2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:16.511510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:16.585535 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1130/2255] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:03:16 2022] PUT /identity/v3/projects/9d1b07435aac450ca6454eb3f9b09686/users/76a57e0a9c954b52afc6f1979c8d02e0/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:03:16.590727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:16.591074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:16.591300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:16.658164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] MFA Rules not processed for user `76a57e0a9c954b52afc6f1979c8d02e0`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:16.659452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:16.823380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1126/2256] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:03:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:17.313117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:18.884401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038bd4f-bf39-43fa-a49a-7ff55e78fe35 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:18.885448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038bd4f-bf39-43fa-a49a-7ff55e78fe35 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:18.885906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038bd4f-bf39-43fa-a49a-7ff55e78fe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:18.914807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f038bd4f-bf39-43fa-a49a-7ff55e78fe35 None None] MFA Rules not processed for user `9c60f6eaccad440abff863fe99b40382`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:18.916574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f038bd4f-bf39-43fa-a49a-7ff55e78fe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:19.019183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1131/2257] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:03:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:19.024315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:19.580681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:19.984322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:20.065187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:23.691340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:23.693687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:23.766706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:23.788229 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5b6bd-194d-427e-8450-1f4c01bbb289 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:23.797196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed50666-7ea9-4372-83bc-2b188faeb392 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:23.798501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50666-7ea9-4372-83bc-2b188faeb392 service glance] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:23.798864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50666-7ea9-4372-83bc-2b188faeb392 service glance] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:23.799405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50666-7ea9-4372-83bc-2b188faeb392 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:23.810598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed50666-7ea9-4372-83bc-2b188faeb392 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:23.812158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed50666-7ea9-4372-83bc-2b188faeb392 service glance] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:23.825850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1127/2258] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:29.999955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1fe91-5fd5-456e-a9fb-2503bda9ba88 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:30.000589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1fe91-5fd5-456e-a9fb-2503bda9ba88 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:30.000589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1fe91-5fd5-456e-a9fb-2503bda9ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:30.024046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-bed1fe91-5fd5-456e-a9fb-2503bda9ba88 None None] MFA Rules not processed for user `026b4316e9fb4f78ae1e9c7c3fa6c201`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:30.025211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bed1fe91-5fd5-456e-a9fb-2503bda9ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:30.111382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1132/2259] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:03:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:03:33.950305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=10 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:33.968439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139bc8a-217a-4c0d-a8cc-bdc2d5b4246e None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:33.968746 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139bc8a-217a-4c0d-a8cc-bdc2d5b4246e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:33.968983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139bc8a-217a-4c0d-a8cc-bdc2d5b4246e None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:33.972606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1128/2260] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:03:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:03:33.986846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122616-1c4a-46f2-9427-62c94559cfb4 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:33.987808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122616-1c4a-46f2-9427-62c94559cfb4 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:33.987808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122616-1c4a-46f2-9427-62c94559cfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.010856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e8122616-1c4a-46f2-9427-62c94559cfb4 None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:34.012458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e8122616-1c4a-46f2-9427-62c94559cfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:34.128741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1133/2261] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:34.145362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139bc8a-217a-4c0d-a8cc-bdc2d5b4246e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:34.149042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e139bc8a-217a-4c0d-a8cc-bdc2d5b4246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:34.168058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139bc8a-217a-4c0d-a8cc-bdc2d5b4246e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:34.172847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb2d4d-789f-46e6-82ec-bca94eebce8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:34.174047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb2d4d-789f-46e6-82ec-bca94eebce8d service glance] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.174666 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb2d4d-789f-46e6-82ec-bca94eebce8d service glance] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.175261 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb2d4d-789f-46e6-82ec-bca94eebce8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.198683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb2d4d-789f-46e6-82ec-bca94eebce8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:34.200034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb2d4d-789f-46e6-82ec-bca94eebce8d service glance] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:34.225522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1129/2262] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:03:34.520136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573a37-6b8b-4c54-8b17-3ef83d90730c None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.520720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573a37-6b8b-4c54-8b17-3ef83d90730c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.521234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573a37-6b8b-4c54-8b17-3ef83d90730c None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.524217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1134/2263] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:03:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 14:03:34.531386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1bdd9-6ef5-4e47-b45b-83bed81df40e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.532748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1bdd9-6ef5-4e47-b45b-83bed81df40e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.533269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1bdd9-6ef5-4e47-b45b-83bed81df40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.560115 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7aa1bdd9-6ef5-4e47-b45b-83bed81df40e None None] MFA Rules not processed for user `b1c427a987ba48878873734ecf0f86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:34.561396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7aa1bdd9-6ef5-4e47-b45b-83bed81df40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:34.633024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1130/2264] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:03:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:34.647648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a573a37-6b8b-4c54-8b17-3ef83d90730c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:34.649827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9a573a37-6b8b-4c54-8b17-3ef83d90730c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:34.667367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a573a37-6b8b-4c54-8b17-3ef83d90730c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:34.671978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d571758-95e8-46fc-bc1b-d3b3e608f4a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:34.672589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571758-95e8-46fc-bc1b-d3b3e608f4a8 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.672868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571758-95e8-46fc-bc1b-d3b3e608f4a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.673170 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571758-95e8-46fc-bc1b-d3b3e608f4a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.681676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d571758-95e8-46fc-bc1b-d3b3e608f4a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:34.683256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d571758-95e8-46fc-bc1b-d3b3e608f4a8 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:34.693503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1135/2265] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:34.828679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe16c-b9fc-489c-a53f-cbad3a1e689a None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.829007 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe16c-b9fc-489c-a53f-cbad3a1e689a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.829232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe16c-b9fc-489c-a53f-cbad3a1e689a None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.830949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1131/2266] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:03:34.840230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a833c77-1e71-4f0b-aef3-0910909b92fd None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.840583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a833c77-1e71-4f0b-aef3-0910909b92fd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.841145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a833c77-1e71-4f0b-aef3-0910909b92fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.869261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-7a833c77-1e71-4f0b-aef3-0910909b92fd None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:34.870641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7a833c77-1e71-4f0b-aef3-0910909b92fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:34.880792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1136/2267] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:03:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:34.892420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fe16c-b9fc-489c-a53f-cbad3a1e689a None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:34.906852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fe16c-b9fc-489c-a53f-cbad3a1e689a None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:34.915346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:34.916346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:34.917058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:34.917307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:34.931897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:34.933378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:34.947411 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1132/2268] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:36.248891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389c11a-5ffa-4588-8e8e-9947a2215c74 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:36.249811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389c11a-5ffa-4588-8e8e-9947a2215c74 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:36.249811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389c11a-5ffa-4588-8e8e-9947a2215c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:36.275519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f389c11a-5ffa-4588-8e8e-9947a2215c74 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:36.278776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f389c11a-5ffa-4588-8e8e-9947a2215c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:36.293650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:36.428567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1137/2269] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:03:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:36.449008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:36.449008 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:36.576585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c3338-7c59-4b57-a9fd-0e5397add436 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:36.590225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:36.590225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:36.590225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:36.590225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:36.599677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:36.601699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:36.615346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1133/2270] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:03:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:03:46.608814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=67 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:46.739422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=21 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:03:50.880904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f37148-74c4-4f0f-b9b2-b2859679ce0b None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:50.881911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f37148-74c4-4f0f-b9b2-b2859679ce0b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:50.882187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:50.882639 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f37148-74c4-4f0f-b9b2-b2859679ce0b None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:50.883745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:50.884947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1138/2271] 10.222.0.61 () {54 vars in 961 bytes} [Thu Sep 1 14:03:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:03:50.894610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e654a79-accc-4fb8-b19e-a7b7755e3d1c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:50.895160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e654a79-accc-4fb8-b19e-a7b7755e3d1c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:50.897372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e654a79-accc-4fb8-b19e-a7b7755e3d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:51.005609 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4d843-890e-41e7-a02a-cd3418044d97 service swift] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:51.015377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:51.016528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:51.017029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:51.017520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:51.021026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:51.037265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:51.039314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:51.054594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1134/2272] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:03:51.086407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-1e654a79-accc-4fb8-b19e-a7b7755e3d1c None None] MFA Rules not processed for user `f9d9a39675444fd9929465b27fb83cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:03:51.089105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1e654a79-accc-4fb8-b19e-a7b7755e3d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:03:51.098046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1139/2273] 10.222.0.61 () {58 vars in 1058 bytes} [Thu Sep 1 14:03:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:03:51.106914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:03:51.121282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb10ed-6659-4d2a-b10c-e9f6baa151ff service placement] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:03:51.129421 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a8bf1-93ee-46e3-90ff-15337f3865e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:03:51.130006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a8bf1-93ee-46e3-90ff-15337f3865e7 service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:03:51.130317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a8bf1-93ee-46e3-90ff-15337f3865e7 service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:03:51.131384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a8bf1-93ee-46e3-90ff-15337f3865e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:03:51.146628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a8bf1-93ee-46e3-90ff-15337f3865e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:03:51.148741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a8bf1-93ee-46e3-90ff-15337f3865e7 service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:03:51.168364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1135/2274] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:03:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:04:01.080384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:01.158256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=38 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:05.391581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e654a79-accc-4fb8-b19e-a7b7755e3d1c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:05.485520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e654a79-accc-4fb8-b19e-a7b7755e3d1c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:05.491094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:05.491736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:05.492091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:05.492345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] PATH_INFO: `/v3/users/76a57e0a9c954b52afc6f1979c8d02e0` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:05.493490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a57e0a9c954b52afc6f1979c8d02e0)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:05.495468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:05.722336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a57e0a9c954b52afc6f1979c8d02e0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:05.741579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a57e0a9c954b52afc6f1979c8d02e0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:05.742565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a57e0a9c954b52afc6f1979c8d02e0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:05.743205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a57e0a9c954b52afc6f1979c8d02e0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:05.823910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a57e0a9c954b52afc6f1979c8d02e0'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:05.864509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1140/2275] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:04:05 2022] DELETE /identity/v3/users/76a57e0a9c954b52afc6f1979c8d02e0 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:04:05.869634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e424a-a17c-4179-aa42-bfe6420d3bc1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:05.870400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e424a-a17c-4179-aa42-bfe6420d3bc1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:05.870400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e424a-a17c-4179-aa42-bfe6420d3bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:05.891914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-138e424a-a17c-4179-aa42-bfe6420d3bc1 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:05.903071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-138e424a-a17c-4179-aa42-bfe6420d3bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:05.978683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1136/2276] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:04:05.989390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:05.991521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:06.054325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:06.054838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:06.055113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:06.077756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:06.081549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:06.098444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9342-c7c7-4894-9ed5-d4329059dfe4 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:06.103139 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:06.103781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:06.104054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:06.104320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:06.114746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:06.121605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:06.127040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1141/2277] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:04:06.152248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1137/2278] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:04:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:06.184953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:06.184953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:06.226408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:06.291963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562ded5-8dd1-462b-83a5-20146c10a49c service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:06.297319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4a0e3-01a8-45e0-a9c9-032bbe8c0f3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:06.297997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4a0e3-01a8-45e0-a9c9-032bbe8c0f3f service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:06.298272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4a0e3-01a8-45e0-a9c9-032bbe8c0f3f service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:06.308663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4a0e3-01a8-45e0-a9c9-032bbe8c0f3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:06.308663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4a0e3-01a8-45e0-a9c9-032bbe8c0f3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:06.312590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4a0e3-01a8-45e0-a9c9-032bbe8c0f3f service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:06.319443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1142/2279] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:04:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:04:07.037337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:07.039680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:07.065043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca8072-2651-4eaa-acf1-0947d08da2f6 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:07.065043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:07.065043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:07.065043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:07.065491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] PATH_INFO: `/v3/projects/9d1b07435aac450ca6454eb3f9b09686` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:07.065491 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d1b07435aac450ca6454eb3f9b09686)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:07.068317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:07.435649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d1b07435aac450ca6454eb3f9b09686', 'request_id': 'req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:07.455771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d1b07435aac450ca6454eb3f9b09686', 'request_id': 'req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:07.555857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d1b07435aac450ca6454eb3f9b09686', 'request_id': 'req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:07.581964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] The token cache is being invalidate because project 9d1b07435aac450ca6454eb3f9b09686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:04:07.582632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:07.583821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1dac5ed3-7fd6-4154-aed3-fc7274f2129e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:07.586991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1138/2280] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:04:07 2022] DELETE /identity/v3/projects/9d1b07435aac450ca6454eb3f9b09686 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:04:07.594787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655628-8e6f-41d7-82c3-f3bc3d72c6b3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:07.595392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655628-8e6f-41d7-82c3-f3bc3d72c6b3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:07.595757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655628-8e6f-41d7-82c3-f3bc3d72c6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:07.637918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-17655628-8e6f-41d7-82c3-f3bc3d72c6b3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:07.639505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-17655628-8e6f-41d7-82c3-f3bc3d72c6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:07.769892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1143/2281] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:07.786800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36331d2c-8de8-462e-9d43-3e148543a13f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:07.787313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36331d2c-8de8-462e-9d43-3e148543a13f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:07.787702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36331d2c-8de8-462e-9d43-3e148543a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:07.812620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-36331d2c-8de8-462e-9d43-3e148543a13f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:07.814111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-36331d2c-8de8-462e-9d43-3e148543a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:07.824474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1139/2282] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:07.829308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655628-8e6f-41d7-82c3-f3bc3d72c6b3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:07.840597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655628-8e6f-41d7-82c3-f3bc3d72c6b3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:07.849041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:07.849718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:07.850035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:07.850366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:07.851179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:04:07.851484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:07.853260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:07.878809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1144/2283] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:04:07.884349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36331d2c-8de8-462e-9d43-3e148543a13f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:07.907616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36331d2c-8de8-462e-9d43-3e148543a13f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:07.912900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:07.913724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:07.914234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:07.914608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:07.915375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:07.917435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:08.039382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:08.049341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1140/2284] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:04:07 2022] POST /identity/v3/projects => generated 369 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:04:08.057590 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:08.083693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7e636-3c0b-4bb4-a67b-90dbed0500ef admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:08.196319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:08.197320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:08.197854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:08.198400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:08.199561 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:08.203160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:08.339046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1145/2285] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:04:08 2022] POST /identity/v3/users => generated 285 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:04:08.343956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:08.369011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc531fb-3aef-4b9e-80de-dd8ed97bf846 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:08.387572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:08.388316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:08.388722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:08.389071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:08.389951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:08.392434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:08.409506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1141/2286] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:04:08.415118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:08.433111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727940c-867f-4fa6-b730-ceb0c0ac27c1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:08.443216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda582b-6c7b-461f-aa34-1cb7860e74fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:08.444025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda582b-6c7b-461f-aa34-1cb7860e74fd admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:08.444302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda582b-6c7b-461f-aa34-1cb7860e74fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:08.444572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda582b-6c7b-461f-aa34-1cb7860e74fd admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:08.445332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda582b-6c7b-461f-aa34-1cb7860e74fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:08.448588 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda582b-6c7b-461f-aa34-1cb7860e74fd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:08.477931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1146/2287] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:04:08.484304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:08.497235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb7c5e-4bf8-4b23-bae5-6819435438db admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:08.503901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:08.505741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:08.506109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:08.506393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] PATH_INFO: `/v3/projects/ddbea428083e4723b6c67d214055a18b/users/a542c969db6e43c899d77c145f4b6e45/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:08.550499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbea428083e4723b6c67d214055a18b, user_id=a542c969db6e43c899d77c145f4b6e45, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:08.560630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:08.678639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1142/2288] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:04:08 2022] PUT /identity/v3/projects/ddbea428083e4723b6c67d214055a18b/users/a542c969db6e43c899d77c145f4b6e45/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:04:08.694474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca092e-23c5-46c0-a6d3-17473a1420fb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:08.696932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca092e-23c5-46c0-a6d3-17473a1420fb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:08.699088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca092e-23c5-46c0-a6d3-17473a1420fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:08.749690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c6ca092e-23c5-46c0-a6d3-17473a1420fb None None] MFA Rules not processed for user `a542c969db6e43c899d77c145f4b6e45`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:08.751132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c6ca092e-23c5-46c0-a6d3-17473a1420fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:08.759667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:08.884976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1147/2289] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:08.900513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:08.916774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:09.014674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ebd7-d42b-4323-9970-6a9fabf7f4e3 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:09.024449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:09.025089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:09.025371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:09.025646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:09.041068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:09.045544 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:09.058144 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1143/2290] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:04:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:04:19.050181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:19.116675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=24 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:27.203734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9018dec-7ea9-426c-9cb1-e46b1f6a508a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:27.205002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9018dec-7ea9-426c-9cb1-e46b1f6a508a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:27.210577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9018dec-7ea9-426c-9cb1-e46b1f6a508a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:27.257023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f9018dec-7ea9-426c-9cb1-e46b1f6a508a None None] MFA Rules not processed for user `65c88d8b86544ab5825adbadb26b1e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:27.258348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f9018dec-7ea9-426c-9cb1-e46b1f6a508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:27.369819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1148/2291] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:28.919441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:28.922127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:29.051477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1cef-854e-4be9-bdd0-80699a5e737f service glance] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:29.056179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:29.057667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:29.058127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:29.063551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:29.063551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:29.154051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:29.156678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:29.169086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1144/2292] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:04:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:04:34.294624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7e4c6-dd2a-481e-8d41-e1596ba33b1e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:34.295838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7e4c6-dd2a-481e-8d41-e1596ba33b1e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:34.296528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7e4c6-dd2a-481e-8d41-e1596ba33b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:34.330793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-bcd7e4c6-dd2a-481e-8d41-e1596ba33b1e None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:34.333601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bcd7e4c6-dd2a-481e-8d41-e1596ba33b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:34.411053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1149/2293] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:04:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:34.423556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:34.426138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:34.522680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413aff-fb4b-4a8c-9d73-4a7057515d2b service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:34.528359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:34.529686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:34.529686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:34.529686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:34.540674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:34.543100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:34.558018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1145/2294] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:04:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:04:36.400835 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea63a-f021-4e4a-953a-b9e8319d4b06 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:36.401809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea63a-f021-4e4a-953a-b9e8319d4b06 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:36.401809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea63a-f021-4e4a-953a-b9e8319d4b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:36.449452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5a6ea63a-f021-4e4a-953a-b9e8319d4b06 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:36.451187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5a6ea63a-f021-4e4a-953a-b9e8319d4b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:36.462158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1150/2295] 10.222.0.61 () {58 vars in 967 bytes} [Thu Sep 1 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:36.489015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:36.507312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9954f9b-c10d-479d-a734-ccc2d4cd68e6 service swift] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:36.521999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a488b-7ad7-4156-abb6-d17974a4a1be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:36.522364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a488b-7ad7-4156-abb6-d17974a4a1be service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:36.523539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a488b-7ad7-4156-abb6-d17974a4a1be service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:36.523539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a488b-7ad7-4156-abb6-d17974a4a1be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:36.543271 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a488b-7ad7-4156-abb6-d17974a4a1be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:36.544883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a488b-7ad7-4156-abb6-d17974a4a1be service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:36.583209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1146/2296] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:04:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:04:46.435789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=53 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:46.560805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=88 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:53.892263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ea63a-f021-4e4a-953a-b9e8319d4b06 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:53.966278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ea63a-f021-4e4a-953a-b9e8319d4b06 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:53.970773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:53.971397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:53.971676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:53.971947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] PATH_INFO: `/v3/users/a542c969db6e43c899d77c145f4b6e45` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:53.973584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a542c969db6e43c899d77c145f4b6e45)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:53.975809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:54.223788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a542c969db6e43c899d77c145f4b6e45'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:54.231409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a542c969db6e43c899d77c145f4b6e45'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:54.232454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a542c969db6e43c899d77c145f4b6e45'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:54.233235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a542c969db6e43c899d77c145f4b6e45'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:54.315074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a542c969db6e43c899d77c145f4b6e45'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:54.346958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c887c5-f5cc-4c5c-aa7f-593ac10a75ef None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:54.348395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c887c5-f5cc-4c5c-aa7f-593ac10a75ef None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:54.348395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c887c5-f5cc-4c5c-aa7f-593ac10a75ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:54.349833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1151/2297] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:04:53 2022] DELETE /identity/v3/users/a542c969db6e43c899d77c145f4b6e45 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:04:54.370922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-46c887c5-f5cc-4c5c-aa7f-593ac10a75ef None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:54.372516 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-46c887c5-f5cc-4c5c-aa7f-593ac10a75ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:54.451386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1147/2298] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:04:54.461857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:54.463259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:54.531922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733eb2f-0292-4735-99b8-78e0be75e6c0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:54.535692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:54.536399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:54.536746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:54.537167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:54.547469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:54.549025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:54.559621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1152/2299] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:04:54.681520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:54.681835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:54.682113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:54.703170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:54.704463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:54.793902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1148/2300] 10.222.0.61 () {58 vars in 967 bytes} [Thu Sep 1 14:04:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:04:54.809161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:54.811280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:54.844319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:54.900579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b76ba-f1ba-4bec-9baf-d67648015929 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:54.907422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:54.912040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:54.912040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:54.912040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:54.920054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:54.921565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:54.932518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1153/2301] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:04:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 14:04:55.136330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:55.136330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:55.152896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacd51e-bf80-46b3-b446-b0a952aa0f77 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:55.158254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:55.159727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:55.160288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:55.160767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] PATH_INFO: `/v3/projects/ddbea428083e4723b6c67d214055a18b` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:55.162108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbea428083e4723b6c67d214055a18b)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:55.164124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:55.403209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbea428083e4723b6c67d214055a18b', 'request_id': 'req-b283e64a-428e-4ab2-b980-20d1dbdef350'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:55.409708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbea428083e4723b6c67d214055a18b', 'request_id': 'req-b283e64a-428e-4ab2-b980-20d1dbdef350'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:55.537925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbea428083e4723b6c67d214055a18b', 'request_id': 'req-b283e64a-428e-4ab2-b980-20d1dbdef350'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:55.549794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] The token cache is being invalidate because project ddbea428083e4723b6c67d214055a18b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:04:55.550117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:55.550748 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:55.553041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1149/2302] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:04:55 2022] DELETE /identity/v3/projects/ddbea428083e4723b6c67d214055a18b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:04:58.736680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:58.740383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:58.766177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acada3-2c9e-4c63-8837-2d4f29416fb8 service swift] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:58.830074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:58.830636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:58.831398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:58.831398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] PATH_INFO: `/v3/users/9c60f6eaccad440abff863fe99b40382` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:58.832930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c60f6eaccad440abff863fe99b40382)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:58.834988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:59.056038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c60f6eaccad440abff863fe99b40382'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.063105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c60f6eaccad440abff863fe99b40382'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.063865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c60f6eaccad440abff863fe99b40382'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.064490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c60f6eaccad440abff863fe99b40382'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.160607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-7e0a2b0f-79db-4882-a7a2-7d5b1d3babc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c60f6eaccad440abff863fe99b40382'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.179264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1154/2303] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:04:58 2022] DELETE /identity/v3/users/9c60f6eaccad440abff863fe99b40382 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:04:59.444268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:59.446578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:59.465694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283e64a-428e-4ab2-b980-20d1dbdef350 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:59.498572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:59.537186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:59.538313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:59.539085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:59.539774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] PATH_INFO: `/v3/projects/596cdff155be4dc997ce351b7cd6db08` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:59.541867 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=596cdff155be4dc997ce351b7cd6db08)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:59.545866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:59.653792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '596cdff155be4dc997ce351b7cd6db08', 'request_id': 'req-d51510a5-12dc-4d3c-8ebd-d089f47589ff'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.659364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '596cdff155be4dc997ce351b7cd6db08', 'request_id': 'req-d51510a5-12dc-4d3c-8ebd-d089f47589ff'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.750032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '596cdff155be4dc997ce351b7cd6db08', 'request_id': 'req-d51510a5-12dc-4d3c-8ebd-d089f47589ff'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.765999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] The token cache is being invalidate because project 596cdff155be4dc997ce351b7cd6db08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:04:59.766372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.767009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d51510a5-12dc-4d3c-8ebd-d089f47589ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:04:59.768750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1150/2304] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:04:59 2022] DELETE /identity/v3/projects/596cdff155be4dc997ce351b7cd6db08 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:04:59.774032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae6cd-c801-4c5c-94fe-82b26289771c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:59.774506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae6cd-c801-4c5c-94fe-82b26289771c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:59.774593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae6cd-c801-4c5c-94fe-82b26289771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:59.795236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6fdae6cd-c801-4c5c-94fe-82b26289771c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:59.796520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6fdae6cd-c801-4c5c-94fe-82b26289771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:59.833677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:04:59.873649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1155/2305] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:04:59.883135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a6a8f-7a2f-400a-9c94-c8375449eacd None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:59.883417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a6a8f-7a2f-400a-9c94-c8375449eacd None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:59.883716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a6a8f-7a2f-400a-9c94-c8375449eacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:59.909308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-671a6a8f-7a2f-400a-9c94-c8375449eacd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:04:59.909308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-671a6a8f-7a2f-400a-9c94-c8375449eacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:04:59.916213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1151/2306] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:04:59.921070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae6cd-c801-4c5c-94fe-82b26289771c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:59.935182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae6cd-c801-4c5c-94fe-82b26289771c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:59.941178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:59.943670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:59.943670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:59.943670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:59.943670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:04:59.943670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:59.946608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:04:59.965044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1156/2307] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:04:59.966958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a6a8f-7a2f-400a-9c94-c8375449eacd None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:04:59.985301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a6a8f-7a2f-400a-9c94-c8375449eacd None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:04:59.990923 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:04:59.991525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:04:59.991819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:04:59.992074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:04:59.992719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:04:59.994705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:00.118272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1152/2308] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:04:59 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:05:00.122621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:00.133938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe2732-cb98-4bb8-a6c2-bc834a5061c6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:00.214752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:00.215521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:00.215803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:00.216135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:00.216826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:00.218857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:00.306436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1157/2309] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:00 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:00.308875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:00.321242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b9084-9de1-43bc-a2fa-6bd82533cc5e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:00.327061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:00.328431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:00.328431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:00.328431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:00.329108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:00.330810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:00.346226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1153/2310] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:05:00.347607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:00.358528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47357c87-2e75-4dea-95c1-197c91956923 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:00.363471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69858a5f-ca17-4887-af1f-6021eca7392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:00.364102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69858a5f-ca17-4887-af1f-6021eca7392c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:00.364374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69858a5f-ca17-4887-af1f-6021eca7392c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:00.364642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69858a5f-ca17-4887-af1f-6021eca7392c admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:00.365337 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69858a5f-ca17-4887-af1f-6021eca7392c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:00.367125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69858a5f-ca17-4887-af1f-6021eca7392c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:00.379280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1158/2311] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:00.396809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:00.408119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af410441-c632-4c72-a137-597e3892fcfd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:00.415091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:00.416153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:00.416371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:00.416660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] PATH_INFO: `/v3/projects/9086ba5c610d4954a238d6569ec32f0b/users/748b8b4771ac498a88fdd98c8a4914ad/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:00.442077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9086ba5c610d4954a238d6569ec32f0b, user_id=748b8b4771ac498a88fdd98c8a4914ad, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:00.447533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:00.553086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1154/2312] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:00 2022] PUT /identity/v3/projects/9086ba5c610d4954a238d6569ec32f0b/users/748b8b4771ac498a88fdd98c8a4914ad/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:00.559904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9b33-6841-4f03-974b-504cf666022f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:00.560464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9b33-6841-4f03-974b-504cf666022f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:00.560993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b9b33-6841-4f03-974b-504cf666022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:00.584012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-181b9b33-6841-4f03-974b-504cf666022f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:00.585612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-181b9b33-6841-4f03-974b-504cf666022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:00.675928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1159/2313] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:00.686055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:00.687956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:00.756149 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb862c32-4eaa-4bea-8526-edd4ed49212f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:00.760402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727797c-4732-4848-9ffe-ae469e5f263f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:00.761092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727797c-4732-4848-9ffe-ae469e5f263f service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:00.761447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727797c-4732-4848-9ffe-ae469e5f263f service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:00.761808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727797c-4732-4848-9ffe-ae469e5f263f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:00.774390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727797c-4732-4848-9ffe-ae469e5f263f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:00.776281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727797c-4732-4848-9ffe-ae469e5f263f service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:00.780983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:00.787680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1155/2314] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:01.375068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b4650-616a-4048-b221-cb69b8896fba None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:01.375608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b4650-616a-4048-b221-cb69b8896fba None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:01.375608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b4650-616a-4048-b221-cb69b8896fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:01.395846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-127b4650-616a-4048-b221-cb69b8896fba None None] MFA Rules not processed for user `c0afa15991f14e0ca0d3701d06a89049`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:01.397064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-127b4650-616a-4048-b221-cb69b8896fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:01.432794 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:01.454976 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1160/2315] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:05:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:08.565673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfa07f-0ec9-4d64-b38a-28425e4da177 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:08.573671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfa07f-0ec9-4d64-b38a-28425e4da177 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:08.573671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfa07f-0ec9-4d64-b38a-28425e4da177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:08.596123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-10bfa07f-0ec9-4d64-b38a-28425e4da177 None None] MFA Rules not processed for user `748b8b4771ac498a88fdd98c8a4914ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:08.599296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-10bfa07f-0ec9-4d64-b38a-28425e4da177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:08.666497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1156/2316] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:08.671129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b4650-616a-4048-b221-cb69b8896fba None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:08.685125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b4650-616a-4048-b221-cb69b8896fba None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:08.695126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:08.696084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:08.697240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:08.698800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:08.699987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:08.704015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:08.805052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1161/2317] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:05:08 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:05:08.808990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfa07f-0ec9-4d64-b38a-28425e4da177 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:08.822990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfa07f-0ec9-4d64-b38a-28425e4da177 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:08.901026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:08.901737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:08.902092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:08.902374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:08.903033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:08.905945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:09.014852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1157/2318] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:05:08 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:05:09.018920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:09.031719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88fbfb-9723-41b4-8254-e922327a5117 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:09.037999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:09.038660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:09.038986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:09.039263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:09.039918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:09.041527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:09.052656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1162/2319] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:05:09.056784 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:09.067968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0bb71-60e3-405e-80cc-ad42b0d5d188 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:09.073377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:09.073896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:09.074177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:09.074450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] PATH_INFO: `/v3/projects/9179b845c5254eeb9d47970a15b462d2/users/920218bc8feb4ff88c2a26febf175360/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:09.091941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9179b845c5254eeb9d47970a15b462d2, user_id=920218bc8feb4ff88c2a26febf175360, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:09.094624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:09.155347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1158/2320] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:09 2022] PUT /identity/v3/projects/9179b845c5254eeb9d47970a15b462d2/users/920218bc8feb4ff88c2a26febf175360/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:05:09.159475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:09.170417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f643d9-e290-407d-8450-5538a74b289d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:09.231084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe215b3-2410-4135-9ad7-24fb43626e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:09.231771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe215b3-2410-4135-9ad7-24fb43626e2e admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:09.232115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe215b3-2410-4135-9ad7-24fb43626e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:09.232512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe215b3-2410-4135-9ad7-24fb43626e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:09.233293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe215b3-2410-4135-9ad7-24fb43626e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:09.235036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe215b3-2410-4135-9ad7-24fb43626e2e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:09.254984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:09.257085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1163/2321] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:05:09.270687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e547-4261-48ca-8877-8ab1774b2bca admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:09.276709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:09.277802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:09.278297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:09.279041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] PATH_INFO: `/v3/projects/9179b845c5254eeb9d47970a15b462d2/users/920218bc8feb4ff88c2a26febf175360/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:09.281808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9179b845c5254eeb9d47970a15b462d2, user_id=920218bc8feb4ff88c2a26febf175360, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:09.284854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:09.299474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1159/2322] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:05:09 2022] PUT /identity/v3/projects/9179b845c5254eeb9d47970a15b462d2/users/920218bc8feb4ff88c2a26febf175360/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:05:15.634877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:15.635758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:15.636138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:15.671150 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] MFA Rules not processed for user `920218bc8feb4ff88c2a26febf175360`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:15.676858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:15.758283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1164/2323] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:05:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:15.772543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:15.774390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:15.819907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:15.855636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de06503-e2b7-45ba-b9f2-ea3f805eba8b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:15.862336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:15.863499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:15.863778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:15.864059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:15.928305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:15.929928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:15.939120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1160/2324] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:25.755192 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:25.935205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=38 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:05:31.543683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:31.545120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:31.608221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aae28-46c0-4731-92ab-387f3c2ff2fa None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:31.611754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce7bdb-4adb-4249-8051-31f793ba7251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:31.612353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7bdb-4adb-4249-8051-31f793ba7251 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:31.612627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7bdb-4adb-4249-8051-31f793ba7251 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:31.612895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7bdb-4adb-4249-8051-31f793ba7251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:31.615646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bdce7bdb-4adb-4249-8051-31f793ba7251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:31.627815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce7bdb-4adb-4249-8051-31f793ba7251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:31.629234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce7bdb-4adb-4249-8051-31f793ba7251 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:31.639961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1165/2325] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:32.839402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:32.850861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae4ef-19f2-4dc0-9a2e-573d5852fd3b service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:32.855782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b76401-54c1-49aa-b77f-ce96638ba4b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:32.855782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76401-54c1-49aa-b77f-ce96638ba4b2 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:32.856265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76401-54c1-49aa-b77f-ce96638ba4b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:32.856548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76401-54c1-49aa-b77f-ce96638ba4b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:32.858716 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f9b76401-54c1-49aa-b77f-ce96638ba4b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:32.873592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b76401-54c1-49aa-b77f-ce96638ba4b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:32.875376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b76401-54c1-49aa-b77f-ce96638ba4b2 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:32.889131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1161/2326] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:05:38.592129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd26e5-7c97-42fd-ba7d-b860a17c96cb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:38.601895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd26e5-7c97-42fd-ba7d-b860a17c96cb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:38.601895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd26e5-7c97-42fd-ba7d-b860a17c96cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:38.614754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c7cd26e5-7c97-42fd-ba7d-b860a17c96cb None None] MFA Rules not processed for user `748b8b4771ac498a88fdd98c8a4914ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:38.616293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c7cd26e5-7c97-42fd-ba7d-b860a17c96cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:38.624263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1166/2327] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:40.619830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:40.621173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:40.621173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:40.647485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] MFA Rules not processed for user `b77f38582b8842d7a303e8431299191a`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:40.648778 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:40.707668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1162/2328] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:46.576072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd26e5-7c97-42fd-ba7d-b860a17c96cb None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:46.591998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd26e5-7c97-42fd-ba7d-b860a17c96cb None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:46.598188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a4a7b-a6e9-4737-9b6f-75c292422ba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:46.598816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a4a7b-a6e9-4737-9b6f-75c292422ba0 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:46.599091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a4a7b-a6e9-4737-9b6f-75c292422ba0 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:46.601283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a4a7b-a6e9-4737-9b6f-75c292422ba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:46.601465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-820a4a7b-a6e9-4737-9b6f-75c292422ba0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:46.697598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a4a7b-a6e9-4737-9b6f-75c292422ba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:46.699511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a4a7b-a6e9-4737-9b6f-75c292422ba0 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:46.709896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1167/2329] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:50.020839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:50.022386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:50.043151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a921f7-3bb6-4156-bad5-604dbff2fbc0 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:50.048776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:50.049454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:50.049739 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:50.050083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:50.052200 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:50.063607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:50.065195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:50.075498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1163/2330] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:51.129898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:51.130544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:51.130544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:51.150771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] MFA Rules not processed for user `1943057b69d342d9b45fc02be9f3d5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:05:51.152018 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:51.163775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1168/2331] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:05:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:05:51.687260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:51.696984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14003c90-7b16-40be-9a43-b63ed781b66d service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:51.700761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:51.701383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:51.701668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:51.701978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:51.703833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:51.714959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:51.716396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:51.725573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1164/2332] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:05:52.468903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:05:52.472074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:52.563606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7feead-e249-4a5c-a35d-c363935bc6ad None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:05:52.581301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:05:52.582343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:05:52.583598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:05:52.583970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:05:52.586068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:05:52.599988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:05:52.601873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:05:52.615498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1169/2333] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:05:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:00.243150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:00.247539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=44 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:00.253075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32297ef7-38f3-4051-84ed-d88fe142e39f service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:00.257074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:00.258668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:00.258946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:00.259216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:00.261107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:00.271535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:00.272959 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:00.282364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1165/2334] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:02.610071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:06.796328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:06.806931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9af448-de0d-483f-90d1-0321223b30a5 service placement] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:06.811560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:06.812280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:06.812446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:06.812677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:06.825939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:06.827317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:06.838650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1170/2335] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:08.208926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:08.225683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43868-d9f9-4683-8056-c89a55754ab6 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:08.232182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:08.270266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:08.270266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:08.270266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:08.270266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:08.319142 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:08.321353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:08.332227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1166/2336] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:13.193240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:13.216413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e045d-bcbb-4b48-a319-49ab7511fec9 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:13.223640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c63211-275c-487e-93b8-7d72277522c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:13.224257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63211-275c-487e-93b8-7d72277522c3 service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:13.224531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63211-275c-487e-93b8-7d72277522c3 service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:13.224798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63211-275c-487e-93b8-7d72277522c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:13.226683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-62c63211-275c-487e-93b8-7d72277522c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:13.241108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c63211-275c-487e-93b8-7d72277522c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:13.260299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c63211-275c-487e-93b8-7d72277522c3 service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:13.260299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1171/2337] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:06:18.327440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=33 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:19.809715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:19.821611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1b446-3c31-4dc5-bbd4-6f65ba31202a service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:19.828235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76e081-fbc8-4562-a75a-767ada25c47a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:19.829335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76e081-fbc8-4562-a75a-767ada25c47a service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:19.830138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76e081-fbc8-4562-a75a-767ada25c47a service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:19.830591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76e081-fbc8-4562-a75a-767ada25c47a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:19.832710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9d76e081-fbc8-4562-a75a-767ada25c47a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:19.850086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76e081-fbc8-4562-a75a-767ada25c47a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:19.852765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76e081-fbc8-4562-a75a-767ada25c47a service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:19.869324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1167/2338] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:23.080138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:23.080715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:23.080815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:23.107187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] MFA Rules not processed for user `65c88d8b86544ab5825adbadb26b1e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:23.108420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:23.167790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1172/2339] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:06:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:29.861562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=6 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:33.157038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=37 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:34.919857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe4033-180f-4d2d-92e7-ff2f67f875e0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:34.920325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe4033-180f-4d2d-92e7-ff2f67f875e0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:34.920432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe4033-180f-4d2d-92e7-ff2f67f875e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:34.940141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ccbe4033-180f-4d2d-92e7-ff2f67f875e0 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:34.941374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ccbe4033-180f-4d2d-92e7-ff2f67f875e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:35.005982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1168/2340] 10.222.0.61 () {58 vars in 967 bytes} [Thu Sep 1 14:06:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:35.140813 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:35.142263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:35.220510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b643e-e301-4a56-a959-f297b17fd175 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:35.225880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:35.227251 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:35.227589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:35.227909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:35.236620 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:35.238248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:35.246314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1173/2341] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:06:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:06:39.859303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae26b10-2454-4fb5-85e9-4615f6ee9a49 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:39.859865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae26b10-2454-4fb5-85e9-4615f6ee9a49 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:39.859865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae26b10-2454-4fb5-85e9-4615f6ee9a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:39.881135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-bae26b10-2454-4fb5-85e9-4615f6ee9a49 None None] MFA Rules not processed for user `c0afa15991f14e0ca0d3701d06a89049`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:06:39.882396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-bae26b10-2454-4fb5-85e9-4615f6ee9a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:39.890166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1169/2342] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:06:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:06:40.385021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:40.398035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115857dc-5855-438a-abf8-56bc0dde18f5 service swift] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:40.399415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:40.400037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:40.400942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:40.401368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:40.403314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:40.414008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:40.415475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:40.425703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1174/2343] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:45.590848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae26b10-2454-4fb5-85e9-4615f6ee9a49 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:45.608478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae26b10-2454-4fb5-85e9-4615f6ee9a49 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:45.614043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:45.615291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:45.615593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:45.616019 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:45.618827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:45.642826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:45.644621 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:45.655861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1170/2344] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:45.756407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:45.758228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:45.833981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7439-5bc5-44a1-a760-dde5076a96ae service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:45.838914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:45.839549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:45.839811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:45.840083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] PATH_INFO: `/v3/users/026b4316e9fb4f78ae1e9c7c3fa6c201` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:45.841239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=026b4316e9fb4f78ae1e9c7c3fa6c201)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:45.843234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:46.027163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026b4316e9fb4f78ae1e9c7c3fa6c201'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.034825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026b4316e9fb4f78ae1e9c7c3fa6c201'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.035554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026b4316e9fb4f78ae1e9c7c3fa6c201'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.036201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026b4316e9fb4f78ae1e9c7c3fa6c201'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.108776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026b4316e9fb4f78ae1e9c7c3fa6c201'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.122225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:46.129658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1175/2345] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:45 2022] DELETE /identity/v3/users/026b4316e9fb4f78ae1e9c7c3fa6c201 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:46.405061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.430435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0592a-cb8c-4c13-a426-ccf669ca3fe5 service placement] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.520538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.521200 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.521573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.521896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] PATH_INFO: `/v3/projects/a9e92afcc6744da393978638b5b986a5` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.523926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e92afcc6744da393978638b5b986a5)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.526145 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:46.738547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e92afcc6744da393978638b5b986a5', 'request_id': 'req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.744652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e92afcc6744da393978638b5b986a5', 'request_id': 'req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.829926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e92afcc6744da393978638b5b986a5', 'request_id': 'req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.845552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] The token cache is being invalidate because project a9e92afcc6744da393978638b5b986a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:46.846166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.846887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:46.850868 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1171/2346] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:46 2022] DELETE /identity/v3/projects/a9e92afcc6744da393978638b5b986a5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:46.855477 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:46.857072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:46.874009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814789-0845-44de-ab42-d0da42741238 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:46.947471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:46.948124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:46.948617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:46.948900 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] PATH_INFO: `/v3/users/248a831de51e4d518f22ad7531e32c93` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:46.950156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=248a831de51e4d518f22ad7531e32c93)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:46.952114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:47.152416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '248a831de51e4d518f22ad7531e32c93'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.160256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '248a831de51e4d518f22ad7531e32c93'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.160978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '248a831de51e4d518f22ad7531e32c93'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.161589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '248a831de51e4d518f22ad7531e32c93'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.234306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '248a831de51e4d518f22ad7531e32c93'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.258002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1176/2347] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:06:46 2022] DELETE /identity/v3/users/248a831de51e4d518f22ad7531e32c93 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:47.539682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:47.541184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:47.555417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957914e-7ea3-4eaf-bcc2-239d38b90dfe admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:47.569964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:47.619244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:47.619891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:47.620172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:47.620449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] PATH_INFO: `/v3/projects/f7b4da64338a4468b1b7d0ac77c61e47` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:47.621646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b4da64338a4468b1b7d0ac77c61e47)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:47.624187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:47.749856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b4da64338a4468b1b7d0ac77c61e47', 'request_id': 'req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.755910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b4da64338a4468b1b7d0ac77c61e47', 'request_id': 'req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.834386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b4da64338a4468b1b7d0ac77c61e47', 'request_id': 'req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.847571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] The token cache is being invalidate because project f7b4da64338a4468b1b7d0ac77c61e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:06:47.848003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.848703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e4a3d23d-2ce2-4b9f-954d-b4b5d1b85450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:06:47.855156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1172/2348] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:06:47 2022] DELETE /identity/v3/projects/f7b4da64338a4468b1b7d0ac77c61e47 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:06:51.524306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:06:51.526478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:51.616094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35f9ac-773a-4466-bd7e-9dfbda7b4e34 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:06:51.625652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:06:51.705543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:06:51.707573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:06:51.707573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:06:51.707573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:06:51.709157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:06:51.722234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:06:51.723853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:06:51.734249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1177/2349] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:06:57.844099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:02.062927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=28 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:15.149756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.150287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.150388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.170693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:15.173901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:15.233711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1173/2350] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:07:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:15.246929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:15.248336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:15.361842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ce990-3886-4f3d-ab0d-94feae863ecf service nova] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:15.373210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef277-a9eb-4da2-847d-4e94ca1a029b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:15.374069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef277-a9eb-4da2-847d-4e94ca1a029b service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:15.374502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef277-a9eb-4da2-847d-4e94ca1a029b service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:15.374872 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef277-a9eb-4da2-847d-4e94ca1a029b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:15.385815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ef277-a9eb-4da2-847d-4e94ca1a029b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:15.388797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ef277-a9eb-4da2-847d-4e94ca1a029b service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:15.402295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1178/2351] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:07:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:07:17.924656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:17.926350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:17.946958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92066e-a200-4e10-8943-142fd3cc84d5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:17.953511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e9191-9969-421b-a543-10eac3931b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:17.954593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9191-9969-421b-a543-10eac3931b82 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:17.955147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9191-9969-421b-a543-10eac3931b82 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:17.955639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9191-9969-421b-a543-10eac3931b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:17.958507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-454e9191-9969-421b-a543-10eac3931b82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:18.067764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e9191-9969-421b-a543-10eac3931b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:18.070270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e9191-9969-421b-a543-10eac3931b82 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:18.081382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1174/2352] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:25.397646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=32 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:28.079919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=53 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:36.962714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c686dd-b2ca-41a8-88da-dfdf73919695 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:36.963190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c686dd-b2ca-41a8-88da-dfdf73919695 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:36.963284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c686dd-b2ca-41a8-88da-dfdf73919695 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:36.964572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1179/2353] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:07:36.970718 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b845-ec26-44ea-8f91-d7c916c4272a None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:36.971251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b845-ec26-44ea-8f91-d7c916c4272a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:36.971251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b845-ec26-44ea-8f91-d7c916c4272a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:36.993515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-8c69b845-ec26-44ea-8f91-d7c916c4272a None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:36.995183 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8c69b845-ec26-44ea-8f91-d7c916c4272a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:37.005398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1175/2354] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:07:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:37.018424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c686dd-b2ca-41a8-88da-dfdf73919695 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:37.020188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-67c686dd-b2ca-41a8-88da-dfdf73919695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:37.082743 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c686dd-b2ca-41a8-88da-dfdf73919695 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:37.087923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a5c3d-da9f-4b8b-aea7-38286f938664 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f33454f4b94b7fb54e83252ce564db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:37.088537 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a5c3d-da9f-4b8b-aea7-38286f938664 service glance] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:37.088817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a5c3d-da9f-4b8b-aea7-38286f938664 service glance] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:37.089082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a5c3d-da9f-4b8b-aea7-38286f938664 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:37.100126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a5c3d-da9f-4b8b-aea7-38286f938664 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:37.101559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a5c3d-da9f-4b8b-aea7-38286f938664 service glance] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:37.112290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1180/2355] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:37.297879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbd4a-41ad-4a31-bb42-b4d810544cba None None] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:37.298205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbd4a-41ad-4a31-bb42-b4d810544cba None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:37.298510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbd4a-41ad-4a31-bb42-b4d810544cba None None] PATH_INFO: `/` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:37.299912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1176/2356] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:07:37.305146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57a582-feed-4a8b-a125-1246ccdf9a0b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:37.305422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57a582-feed-4a8b-a125-1246ccdf9a0b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:37.305698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57a582-feed-4a8b-a125-1246ccdf9a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:37.326254 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6a57a582-feed-4a8b-a125-1246ccdf9a0b None None] MFA Rules not processed for user `b1c427a987ba48878873734ecf0f86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:37.327440 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6a57a582-feed-4a8b-a125-1246ccdf9a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:37.378836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1181/2357] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:07:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:37.391268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedbd4a-41ad-4a31-bb42-b4d810544cba None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:37.392962 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5aedbd4a-41ad-4a31-bb42-b4d810544cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:37.406508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedbd4a-41ad-4a31-bb42-b4d810544cba None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:37.409987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9001ae-901c-4a36-b642-656ed2d0474d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:37.410571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9001ae-901c-4a36-b642-656ed2d0474d service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:37.410845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9001ae-901c-4a36-b642-656ed2d0474d service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:37.414741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9001ae-901c-4a36-b642-656ed2d0474d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:37.418299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9001ae-901c-4a36-b642-656ed2d0474d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:37.419699 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9001ae-901c-4a36-b642-656ed2d0474d service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:37.427549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1177/2358] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:07:37.510691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187a8e-f940-4738-b045-cdf5af0adfb3 None None] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:37.510971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187a8e-f940-4738-b045-cdf5af0adfb3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:37.511241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187a8e-f940-4738-b045-cdf5af0adfb3 None None] PATH_INFO: `/` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:37.512581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1182/2359] 10.222.0.61 () {54 vars in 921 bytes} [Thu Sep 1 14:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 14:07:37.517464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a53da-9693-4d7c-b9a6-cee8b983474d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:37.517752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a53da-9693-4d7c-b9a6-cee8b983474d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:37.518070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a53da-9693-4d7c-b9a6-cee8b983474d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:37.538334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b56a53da-9693-4d7c-b9a6-cee8b983474d None None] MFA Rules not processed for user `19a91e958d514c0d8a61289bbe3ea63d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:37.539570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b56a53da-9693-4d7c-b9a6-cee8b983474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:37.548953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1178/2360] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Sep 1 14:07:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:37.559919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb187a8e-f940-4738-b045-cdf5af0adfb3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:37.570827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb187a8e-f940-4738-b045-cdf5af0adfb3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:37.574351 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:37.574962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:37.575239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:37.575508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:37.585508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:37.586969 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:37.597381 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1183/2361] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:41.779047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:41.779635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:41.779635 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:41.800281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:41.801519 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:41.809899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1179/2362] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:07:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:41.832485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:41.844525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc6b1c-9871-4d85-bed7-20357f9952c8 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:41.852916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c927906-a755-41ea-82fc-9edf93b4511e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:41.853580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c927906-a755-41ea-82fc-9edf93b4511e service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:41.853892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c927906-a755-41ea-82fc-9edf93b4511e service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:41.854171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c927906-a755-41ea-82fc-9edf93b4511e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:41.866458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c927906-a755-41ea-82fc-9edf93b4511e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:41.866663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c927906-a755-41ea-82fc-9edf93b4511e service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:41.875610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1184/2363] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:07:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:07:51.794129 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=15 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:51.872008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=61 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:07:54.182953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:54.184511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:54.196484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64e92-14c3-4dc8-8681-9d5b893c0dd4 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:54.200273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:54.200895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:54.201174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:54.201449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:54.203392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:54.281911 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:54.283363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:54.298439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1180/2364] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:57.952465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f23c-229d-498e-880a-3d47d482dc57 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:57.952977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f23c-229d-498e-880a-3d47d482dc57 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:57.954248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8f23c-229d-498e-880a-3d47d482dc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:57.975531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-7db8f23c-229d-498e-880a-3d47d482dc57 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:07:57.976774 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7db8f23c-229d-498e-880a-3d47d482dc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:57.986069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1185/2365] 10.222.0.61 () {58 vars in 967 bytes} [Thu Sep 1 14:07:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:07:57.998551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:58.014244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1491e69-d92b-4dc1-8ba7-19bbc2e5d599 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:58.019179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:58.019816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:58.020248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:58.021096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:58.040255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:58.040255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:58.040255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1181/2366] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:07:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:07:58.614866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8f23c-229d-498e-880a-3d47d482dc57 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:58.624114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8f23c-229d-498e-880a-3d47d482dc57 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:58.627905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:58.628580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:58.628881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:58.629149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:58.631547 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:58.643641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:58.645541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:58.663136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1186/2367] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:07:59.279934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:07:59.281505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:07:59.297039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3e3d-48a0-45a5-b98c-8c2acf1c3705 service swift] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:07:59.301857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:07:59.302488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:07:59.302920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:07:59.303202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] PATH_INFO: `/v3/users/65c88d8b86544ab5825adbadb26b1e2e` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:07:59.304370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c88d8b86544ab5825adbadb26b1e2e)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:07:59.306337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:07:59.769540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c88d8b86544ab5825adbadb26b1e2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:59.782180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c88d8b86544ab5825adbadb26b1e2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:59.953901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c88d8b86544ab5825adbadb26b1e2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:59.954712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c88d8b86544ab5825adbadb26b1e2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:59.969745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6b13fc3e-732e-4279-ac0f-b6706298c3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c88d8b86544ab5825adbadb26b1e2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:07:59.976594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1182/2368] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:07:59 2022] DELETE /identity/v3/users/65c88d8b86544ab5825adbadb26b1e2e => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:00.418586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:00.430850 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4cb0b-7cc7-4841-8596-c4ff1d15ac0c service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:00.515340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:00.515905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:00.516184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:00.516458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] PATH_INFO: `/v3/projects/dce10589b8b240e6829f2e158415a1b8` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:00.517700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce10589b8b240e6829f2e158415a1b8)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:00.519764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:00.697314 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce10589b8b240e6829f2e158415a1b8', 'request_id': 'req-11548b25-63be-49e4-9452-5e9779521d73'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:00.704062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce10589b8b240e6829f2e158415a1b8', 'request_id': 'req-11548b25-63be-49e4-9452-5e9779521d73'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:00.710660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce10589b8b240e6829f2e158415a1b8', 'request_id': 'req-11548b25-63be-49e4-9452-5e9779521d73'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:00.870578 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] The token cache is being invalidate because project dce10589b8b240e6829f2e158415a1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:00.870932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:00.874592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-11548b25-63be-49e4-9452-5e9779521d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:00.877493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1187/2369] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:00 2022] DELETE /identity/v3/projects/dce10589b8b240e6829f2e158415a1b8 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:00.882528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082c43-cf9b-4d2a-810b-193d64314666 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:00.882997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082c43-cf9b-4d2a-810b-193d64314666 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:00.883081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082c43-cf9b-4d2a-810b-193d64314666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:00.938837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-00082c43-cf9b-4d2a-810b-193d64314666 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:00.940430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-00082c43-cf9b-4d2a-810b-193d64314666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:01.024674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:01.041169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1183/2370] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:01.049383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce981b-9e60-4b86-8cbd-a99cf864b05c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.049674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce981b-9e60-4b86-8cbd-a99cf864b05c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.049983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce981b-9e60-4b86-8cbd-a99cf864b05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.079152 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b3ce981b-9e60-4b86-8cbd-a99cf864b05c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:01.080680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b3ce981b-9e60-4b86-8cbd-a99cf864b05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:01.091668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1188/2371] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:01.096170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00082c43-cf9b-4d2a-810b-193d64314666 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:01.108965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00082c43-cf9b-4d2a-810b-193d64314666 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:01.114870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:01.115497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.115961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.116337 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.117315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:01.117617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:01.122823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:01.138146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1184/2372] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:01.142486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce981b-9e60-4b86-8cbd-a99cf864b05c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:01.153446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce981b-9e60-4b86-8cbd-a99cf864b05c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:01.161320 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:01.161977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.162264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.162542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.163234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:01.165191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:01.302134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1189/2373] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:01 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:01.305105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:01.324302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e81b1-5f79-413b-964b-e3e9bec3842b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:01.396012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:01.396634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.401456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.401456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.401456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:01.401456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:01.578335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1185/2374] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:01 2022] POST /identity/v3/users => generated 287 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:01.584065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:01.595289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d9084-6923-4fe3-a8d5-7e1045ff47ed admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:01.600220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:01.600870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.601151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.601423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.602138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:01.603798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:01.615354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1190/2375] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:01.619660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:01.630905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540752a6-ef8c-46cc-aab0-d08278eccbb4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:01.635693 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e16d7-f740-474c-bf81-bb6aa5a815c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:01.636307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e16d7-f740-474c-bf81-bb6aa5a815c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.636616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e16d7-f740-474c-bf81-bb6aa5a815c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.636889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e16d7-f740-474c-bf81-bb6aa5a815c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.637628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e16d7-f740-474c-bf81-bb6aa5a815c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:01.639277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e16d7-f740-474c-bf81-bb6aa5a815c6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:01.650809 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1186/2376] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:01.655181 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:01.668980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6311f-1279-4ccd-a9ba-b3f87e9abdf7 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:01.674009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:01.674686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.674962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.675232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] PATH_INFO: `/v3/projects/5515936bd78b4ddd846a99798355deda/users/d4618725aa284e90b5f77c1a6438ea35/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.702880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5515936bd78b4ddd846a99798355deda, user_id=d4618725aa284e90b5f77c1a6438ea35, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:01.705929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:01.869119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1191/2377] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:01 2022] PUT /identity/v3/projects/5515936bd78b4ddd846a99798355deda/users/d4618725aa284e90b5f77c1a6438ea35/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:01.879823 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb44d-1857-4c91-bc0f-c3ab7ee056b9 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:01.880105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb44d-1857-4c91-bc0f-c3ab7ee056b9 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:01.880378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb44d-1857-4c91-bc0f-c3ab7ee056b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:01.914170 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3fafb44d-1857-4c91-bc0f-c3ab7ee056b9 None None] MFA Rules not processed for user `d4618725aa284e90b5f77c1a6438ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:01.915517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3fafb44d-1857-4c91-bc0f-c3ab7ee056b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:01.987085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1187/2378] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:01.992298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:02.001723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:02.061537 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b91ea-c7a9-4a91-8e49-334e693d83bc admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:02.066373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:02.067184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:02.067359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:02.067735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:02.068625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:02.071544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:02.240851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1192/2379] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:01 2022] POST /identity/v3/projects => generated 373 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:02.240957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafb44d-1857-4c91-bc0f-c3ab7ee056b9 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:02.251081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafb44d-1857-4c91-bc0f-c3ab7ee056b9 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:02.309369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:02.310018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:02.310299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:02.310594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:02.311236 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:02.313182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:02.548234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1188/2380] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:02 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:02.552368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:02.564700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c9a9a-49cf-49af-8c3c-5fcecbf4c9f2 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:02.574674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:02.575949 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:02.576374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:02.576724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:02.577453 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:02.579184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:02.592891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1193/2381] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:02.597260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:02.609554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c49f-5236-46ac-bd89-5ce3a0260c3e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:02.615859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:02.616987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:02.617503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:02.617966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] PATH_INFO: `/v3/projects/911cf2a819444b4798d2f9515a6fd3f5/users/7d42ceb527f64f54a5ea87f4315c8711/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:02.636644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=911cf2a819444b4798d2f9515a6fd3f5, user_id=7d42ceb527f64f54a5ea87f4315c8711, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:02.639627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:02.778905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1189/2382] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:02 2022] PUT /identity/v3/projects/911cf2a819444b4798d2f9515a6fd3f5/users/7d42ceb527f64f54a5ea87f4315c8711/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:02.783226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:02.796870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f1d0e-04ae-4f3a-b9b7-b0b574b450b8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:02.937658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa415-7cf9-470e-8ff5-6d0707d93c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:02.940144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa415-7cf9-470e-8ff5-6d0707d93c00 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:02.940643 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa415-7cf9-470e-8ff5-6d0707d93c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:02.941082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa415-7cf9-470e-8ff5-6d0707d93c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:02.942817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfa415-7cf9-470e-8ff5-6d0707d93c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:02.947385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfa415-7cf9-470e-8ff5-6d0707d93c00 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:02.965142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1194/2383] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:02.970598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:02.981020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca103601-d490-4ba7-8cc8-beb59b3c8fd6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:02.988134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:02.989813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:02.990090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:02.990415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] PATH_INFO: `/v3/projects/911cf2a819444b4798d2f9515a6fd3f5/users/7d42ceb527f64f54a5ea87f4315c8711/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:02.992662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=911cf2a819444b4798d2f9515a6fd3f5, user_id=7d42ceb527f64f54a5ea87f4315c8711, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:03.002338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:03.016250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1190/2384] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:02 2022] PUT /identity/v3/projects/911cf2a819444b4798d2f9515a6fd3f5/users/7d42ceb527f64f54a5ea87f4315c8711/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:03.031052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:03.031342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:03.031615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:03.064036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] MFA Rules not processed for user `7d42ceb527f64f54a5ea87f4315c8711`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:03.064036 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:03.092001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:03.152844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1195/2385] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:03.164964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:03.166775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:03.225264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9bd68-c7de-4177-bce7-dfc0be6defcc admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:03.229137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:03.229943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:03.230275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:03.230613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:03.241380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:03.241993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:03.243008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:03.243685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:03.255034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1191/2386] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:03.318522 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f66a6-3155-44c5-8e82-87ed7ab30d3f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:03.323289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:03.323892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:03.324171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:03.324436 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] PATH_INFO: `/v3/users/1943057b69d342d9b45fc02be9f3d5aa` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:03.325585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1943057b69d342d9b45fc02be9f3d5aa)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:03.327589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:03.778689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1943057b69d342d9b45fc02be9f3d5aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:03.785711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1943057b69d342d9b45fc02be9f3d5aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:03.787066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1943057b69d342d9b45fc02be9f3d5aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:03.787876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1943057b69d342d9b45fc02be9f3d5aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:03.976856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-352d6c0c-dd25-49c4-b17f-501fb598a1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1943057b69d342d9b45fc02be9f3d5aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:03.996459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1196/2387] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:03 2022] DELETE /identity/v3/users/1943057b69d342d9b45fc02be9f3d5aa => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:04.246486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:04.258692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5786c6-bcf3-4184-99d1-011b22cd9e0d service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:04.338963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:04.339602 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:04.339880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:04.340151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] PATH_INFO: `/v3/projects/8352a8d484e24264a536b8960dab338f` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:04.341515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8352a8d484e24264a536b8960dab338f)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:04.343447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:04.543793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8352a8d484e24264a536b8960dab338f', 'request_id': 'req-40f5105d-6ddf-451f-816e-6c93ed8f6f84'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.549536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8352a8d484e24264a536b8960dab338f', 'request_id': 'req-40f5105d-6ddf-451f-816e-6c93ed8f6f84'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.679546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:04.747838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8352a8d484e24264a536b8960dab338f', 'request_id': 'req-40f5105d-6ddf-451f-816e-6c93ed8f6f84'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.766783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] The token cache is being invalidate because project 8352a8d484e24264a536b8960dab338f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:04.766783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.766783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-40f5105d-6ddf-451f-816e-6c93ed8f6f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:04.766783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1192/2388] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:04 2022] DELETE /identity/v3/projects/8352a8d484e24264a536b8960dab338f => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:04.770770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a457-0068-4517-9c49-49f12c81651c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:04.771123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a457-0068-4517-9c49-49f12c81651c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:04.771404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a457-0068-4517-9c49-49f12c81651c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:04.792058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d8b5a457-0068-4517-9c49-49f12c81651c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:04.793270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d8b5a457-0068-4517-9c49-49f12c81651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:04.811130 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:04.852937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1197/2389] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:04.867660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e614231-179d-41a2-ae14-d22bee92eb66 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:04.867942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e614231-179d-41a2-ae14-d22bee92eb66 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:04.868217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e614231-179d-41a2-ae14-d22bee92eb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:04.889168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-4e614231-179d-41a2-ae14-d22bee92eb66 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:04.890415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4e614231-179d-41a2-ae14-d22bee92eb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:04.898555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1193/2390] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:04.902811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5a457-0068-4517-9c49-49f12c81651c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:04.913273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5a457-0068-4517-9c49-49f12c81651c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:04.921235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30898850-5934-4038-8cdb-022770003255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:04.921894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30898850-5934-4038-8cdb-022770003255 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:04.922175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30898850-5934-4038-8cdb-022770003255 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:04.922450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30898850-5934-4038-8cdb-022770003255 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:04.923105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30898850-5934-4038-8cdb-022770003255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:04.923437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30898850-5934-4038-8cdb-022770003255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:04.925079 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30898850-5934-4038-8cdb-022770003255 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:04.937493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1198/2391] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:04.943944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e614231-179d-41a2-ae14-d22bee92eb66 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:04.953398 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e614231-179d-41a2-ae14-d22bee92eb66 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:04.958360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:04.959013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:04.959293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:04.959702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:04.960386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:04.962367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:04.994274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30898850-5934-4038-8cdb-022770003255 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:04.996162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-30898850-5934-4038-8cdb-022770003255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.009753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30898850-5934-4038-8cdb-022770003255 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.014350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.015004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.015287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.015562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] PATH_INFO: `/v3/users/d4618725aa284e90b5f77c1a6438ea35` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.016696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4618725aa284e90b5f77c1a6438ea35)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.018660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.126382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1194/2392] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:04 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:05.130708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.191015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4891a27-91bb-4312-b1ef-06f372bbdbce admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.195845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.196446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.196723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.196992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.197694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.199597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.291593 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1195/2393] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:05 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:05.296032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.299617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4618725aa284e90b5f77c1a6438ea35'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:05.305993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95e8c8-a4ba-4b6b-88d5-3b2f2b72cbc6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.306688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4618725aa284e90b5f77c1a6438ea35'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:05.307423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4618725aa284e90b5f77c1a6438ea35'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:05.308059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4618725aa284e90b5f77c1a6438ea35'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:05.312310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.312310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.312310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.312657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.313226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.314907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.325377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1196/2394] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:05.329475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.331188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.344117 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb713-1d97-4262-99a4-9a9e2b068743 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.349456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.350069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.350346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.350616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.351252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.352856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.363249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1197/2395] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:05.367331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.376676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74e5a-bf16-4487-9e27-aab0bf297cda admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.382036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c140e-66dd-4f2e-bd14-f6bf1d99ff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.382660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c140e-66dd-4f2e-bd14-f6bf1d99ff16 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.382935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c140e-66dd-4f2e-bd14-f6bf1d99ff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.383203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c140e-66dd-4f2e-bd14-f6bf1d99ff16 admin admin] PATH_INFO: `/v3/projects/11e0967fff7047b8a06b72c74e0dd45b/users/f80142c5a42d4d94bed8ada35ae48ab3/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.397850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c140e-66dd-4f2e-bd14-f6bf1d99ff16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e0967fff7047b8a06b72c74e0dd45b, user_id=f80142c5a42d4d94bed8ada35ae48ab3, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.400445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c140e-66dd-4f2e-bd14-f6bf1d99ff16 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.473146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5aaff6f2-f211-416d-996a-c1efff32dcd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4618725aa284e90b5f77c1a6438ea35'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:05.491412 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1199/2396] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:04 2022] DELETE /identity/v3/users/d4618725aa284e90b5f77c1a6438ea35 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:05.495756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93185299-b954-4400-ad4c-3c408c0348db None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.496043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93185299-b954-4400-ad4c-3c408c0348db None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.496318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93185299-b954-4400-ad4c-3c408c0348db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.515524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-93185299-b954-4400-ad4c-3c408c0348db None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:05.516721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-93185299-b954-4400-ad4c-3c408c0348db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.574038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1200/2397] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:05.584153 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93185299-b954-4400-ad4c-3c408c0348db None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.585461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-93185299-b954-4400-ad4c-3c408c0348db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.616811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1198/2398] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:05 2022] PUT /identity/v3/projects/11e0967fff7047b8a06b72c74e0dd45b/users/f80142c5a42d4d94bed8ada35ae48ab3/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:05.628705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95b254-01d5-4613-bb6f-8eee061e8e6e None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.628993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95b254-01d5-4613-bb6f-8eee061e8e6e None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.629268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95b254-01d5-4613-bb6f-8eee061e8e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.643391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93185299-b954-4400-ad4c-3c408c0348db None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.658845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-1e95b254-01d5-4613-bb6f-8eee061e8e6e None None] MFA Rules not processed for user `f80142c5a42d4d94bed8ada35ae48ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:05.660122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1e95b254-01d5-4613-bb6f-8eee061e8e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:05.693741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.694384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.694658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.694921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.732198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1199/2399] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:05.736607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95b254-01d5-4613-bb6f-8eee061e8e6e None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.760989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.763061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.774539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1201/2400] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:05.805471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95b254-01d5-4613-bb6f-8eee061e8e6e None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:05.812184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:05.817963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:05.817963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:05.817963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:05.817963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:05.817963 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:05.844944 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:05.960157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1200/2401] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:05 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:05.964344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:05.976383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7b021-a197-4f4c-8a9b-cffc4bed43e5 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.038425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.039167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.039844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.045978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.045978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.045978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.147856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:06.153730 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1202/2402] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:05 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:06.157839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.169185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7fbd7-1e42-44b2-84d0-92df33c8c116 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.173941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.174553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.174831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.175108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.175897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.178140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.188201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1201/2403] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:06.192563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.202755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1319895-99db-4189-aa7c-33ad87814532 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.208871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.209686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.217020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.217020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] PATH_INFO: `/v3/projects/4c6d41041a31464db2da994ebc36b4d9/users/e55c21a975a24d528285230988ade66c/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.226340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6d41041a31464db2da994ebc36b4d9, user_id=e55c21a975a24d528285230988ade66c, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.229316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.245497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.247369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:06.260582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526abad-9000-4270-adf0-3a421d6bfe98 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.265342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.266043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.268935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.268935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] PATH_INFO: `/v3/projects/5515936bd78b4ddd846a99798355deda` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.268935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5515936bd78b4ddd846a99798355deda)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.269641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.367096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1203/2404] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:06 2022] PUT /identity/v3/projects/4c6d41041a31464db2da994ebc36b4d9/users/e55c21a975a24d528285230988ade66c/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:06.370778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.401529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5515936bd78b4ddd846a99798355deda', 'request_id': 'req-fed452bf-127a-4c63-8d95-dba799eda515'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.408249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5515936bd78b4ddd846a99798355deda', 'request_id': 'req-fed452bf-127a-4c63-8d95-dba799eda515'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.436723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c04a-2cec-4b13-9d63-2480d2a253a9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.478814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5515936bd78b4ddd846a99798355deda', 'request_id': 'req-fed452bf-127a-4c63-8d95-dba799eda515'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.492468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] The token cache is being invalidate because project 5515936bd78b4ddd846a99798355deda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:06.493016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.493750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.495548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1202/2405] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:06 2022] DELETE /identity/v3/projects/5515936bd78b4ddd846a99798355deda => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:06.498465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.498465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.498465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.498465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.498882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.499096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.500232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.501902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:06.510226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1204/2406] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:06.514344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.516667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:06.518863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed452bf-127a-4c63-8d95-dba799eda515 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.524306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.524943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.525226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.525504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] PATH_INFO: `/v3/users/7d42ceb527f64f54a5ea87f4315c8711` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.526700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d42ceb527f64f54a5ea87f4315c8711)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.528619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.530323 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cb762-ef2a-40f7-9c18-54e3b49e65a5 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.537218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af233c-0f8d-481b-966d-9fef16df6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.537912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af233c-0f8d-481b-966d-9fef16df6c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.538191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af233c-0f8d-481b-966d-9fef16df6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.538462 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af233c-0f8d-481b-966d-9fef16df6c6a admin admin] PATH_INFO: `/v3/projects/4c6d41041a31464db2da994ebc36b4d9/users/e55c21a975a24d528285230988ade66c/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.541097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af233c-0f8d-481b-966d-9fef16df6c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6d41041a31464db2da994ebc36b4d9, user_id=e55c21a975a24d528285230988ade66c, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.545038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af233c-0f8d-481b-966d-9fef16df6c6a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.557486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1205/2407] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:06 2022] PUT /identity/v3/projects/4c6d41041a31464db2da994ebc36b4d9/users/e55c21a975a24d528285230988ade66c/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:06.564998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:06.567846 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.568393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.568672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.597231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] MFA Rules not processed for user `e55c21a975a24d528285230988ade66c`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:06.599199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:06.678425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1206/2408] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:06.690569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.692143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:06.749175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acab78-85d0-486f-bb2e-2591c1c6fc0d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.753388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.754056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.754332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.754635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.763698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.765212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.775268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1207/2409] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:06.791377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d42ceb527f64f54a5ea87f4315c8711'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.801906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d42ceb527f64f54a5ea87f4315c8711'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.805459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:06.883129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83a9b-3078-4778-a8ba-ef9fa7f59804 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:06.888982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:06.889975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:06.890360 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:06.890735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] PATH_INFO: `/v3/users/f80142c5a42d4d94bed8ada35ae48ab3` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:06.892021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f80142c5a42d4d94bed8ada35ae48ab3)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:06.894133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:06.897379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d42ceb527f64f54a5ea87f4315c8711'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.898296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d42ceb527f64f54a5ea87f4315c8711'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.924331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0f71e074-284a-4190-8e25-b7186ae8f70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d42ceb527f64f54a5ea87f4315c8711'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:06.930865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1203/2410] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:06 2022] DELETE /identity/v3/users/7d42ceb527f64f54a5ea87f4315c8711 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:07.276524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:07.294950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f80142c5a42d4d94bed8ada35ae48ab3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:07.302633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f80142c5a42d4d94bed8ada35ae48ab3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:07.303443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f80142c5a42d4d94bed8ada35ae48ab3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:07.304133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f80142c5a42d4d94bed8ada35ae48ab3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:07.570785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f80142c5a42d4d94bed8ada35ae48ab3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:07.589328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1208/2411] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:06 2022] DELETE /identity/v3/users/f80142c5a42d4d94bed8ada35ae48ab3 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:07.593961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:07.594484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:07.594700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:07.614735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:07.616012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:07.641095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:07.643329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:07.680336 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1204/2412] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:07.699324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:07.699324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:07.742865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d3156-c105-4f4e-8184-730444c107d7 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:07.747681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:07.748397 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:07.748750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:07.749122 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] PATH_INFO: `/v3/projects/911cf2a819444b4798d2f9515a6fd3f5` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:07.750600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=911cf2a819444b4798d2f9515a6fd3f5)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:07.752707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:07.763174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c9a1c-90f8-4c86-a285-f178014f1abf None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:07.773179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34399-7325-4de5-8910-c12cdbeb8cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:07.773179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34399-7325-4de5-8910-c12cdbeb8cbe service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:07.773179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34399-7325-4de5-8910-c12cdbeb8cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:07.773179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34399-7325-4de5-8910-c12cdbeb8cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:07.780914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c34399-7325-4de5-8910-c12cdbeb8cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:07.782505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c34399-7325-4de5-8910-c12cdbeb8cbe service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:07.793012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1205/2413] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:07.948937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '911cf2a819444b4798d2f9515a6fd3f5', 'request_id': 'req-88e98491-8125-4a1b-b305-ee2591d18ef5'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:07.955760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '911cf2a819444b4798d2f9515a6fd3f5', 'request_id': 'req-88e98491-8125-4a1b-b305-ee2591d18ef5'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:07.964028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '911cf2a819444b4798d2f9515a6fd3f5', 'request_id': 'req-88e98491-8125-4a1b-b305-ee2591d18ef5'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.088107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] The token cache is being invalidate because project 911cf2a819444b4798d2f9515a6fd3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:08.088107 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.088711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-88e98491-8125-4a1b-b305-ee2591d18ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.090644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1209/2414] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:07 2022] DELETE /identity/v3/projects/911cf2a819444b4798d2f9515a6fd3f5 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:08.100039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ede9-fc29-4913-859a-020de7058165 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.100428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ede9-fc29-4913-859a-020de7058165 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.100769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ede9-fc29-4913-859a-020de7058165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.131283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b141ede9-fc29-4913-859a-020de7058165 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:08.132734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b141ede9-fc29-4913-859a-020de7058165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:08.218775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1206/2415] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:08.233302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2faafc-bb66-424f-897a-8799c452c1d3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.234014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2faafc-bb66-424f-897a-8799c452c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.234429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2faafc-bb66-424f-897a-8799c452c1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.256324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2a2faafc-bb66-424f-897a-8799c452c1d3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:08.257441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2a2faafc-bb66-424f-897a-8799c452c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:08.265501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1210/2416] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:08.270573 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141ede9-fc29-4913-859a-020de7058165 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.281050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141ede9-fc29-4913-859a-020de7058165 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.286570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.287221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.287500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.287799 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.288452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:08.288783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.290416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.302279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1207/2417] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:08.306739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2faafc-bb66-424f-897a-8799c452c1d3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.316726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2faafc-bb66-424f-897a-8799c452c1d3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.321364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.326840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.326840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.326840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.326840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.326840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.343626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:08.351441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.353372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:08.369955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f0a07-d2a4-48dc-8c87-83f5d6f47990 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.375914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.376640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.376759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.377033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] PATH_INFO: `/v3/projects/11e0967fff7047b8a06b72c74e0dd45b` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.378814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e0967fff7047b8a06b72c74e0dd45b)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.381901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.498445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1211/2418] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:08 2022] POST /identity/v3/projects => generated 363 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:08.499876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.515589 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e0967fff7047b8a06b72c74e0dd45b', 'request_id': 'req-90e7ceea-661b-474c-b9ae-ab37fefec930'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.522134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e0967fff7047b8a06b72c74e0dd45b', 'request_id': 'req-90e7ceea-661b-474c-b9ae-ab37fefec930'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.565852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96535e2-568f-4a68-9e32-072c4cb04613 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.571085 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.571689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.571967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.572289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.572902 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.574859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.644892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e0967fff7047b8a06b72c74e0dd45b', 'request_id': 'req-90e7ceea-661b-474c-b9ae-ab37fefec930'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.658840 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] The token cache is being invalidate because project 11e0967fff7047b8a06b72c74e0dd45b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:08.659139 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.660423 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:08.662194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1208/2419] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:08 2022] DELETE /identity/v3/projects/11e0967fff7047b8a06b72c74e0dd45b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:08.666359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.667706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:08.700073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1212/2420] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:08 2022] POST /identity/v3/users => generated 282 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:08.701398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.711838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:08.716026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb64a9d-5092-495f-ba54-9ef0b0b1e37b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.720766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.721368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.721647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.721959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.722606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.724257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.732162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7ceea-661b-474c-b9ae-ab37fefec930 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.734574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1213/2421] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:08.738593 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.746083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.746083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.746083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.746083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] PATH_INFO: `/v3/users/e55c21a975a24d528285230988ade66c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.746468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e55c21a975a24d528285230988ade66c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.746468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.748663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591249-b536-4072-a016-aa7cce8743fc admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.754134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.755280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.755554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.755824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.756493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.758165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.769081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1214/2422] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:08.773694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:08.783566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46ea22-79fa-416e-9a6e-9cc4bc4f47a8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:08.788258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a718447-ee36-4a88-9b4f-b010e04175f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:08.788896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a718447-ee36-4a88-9b4f-b010e04175f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.789169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a718447-ee36-4a88-9b4f-b010e04175f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.789437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a718447-ee36-4a88-9b4f-b010e04175f7 admin admin] PATH_INFO: `/v3/projects/85e47a815aea43da82c2a2f518511e7a/users/ec863386de134fadad80e1e9101a8798/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.804154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a718447-ee36-4a88-9b4f-b010e04175f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e47a815aea43da82c2a2f518511e7a, user_id=ec863386de134fadad80e1e9101a8798, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:08.806823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a718447-ee36-4a88-9b4f-b010e04175f7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:08.959746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1215/2423] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:08 2022] PUT /identity/v3/projects/85e47a815aea43da82c2a2f518511e7a/users/ec863386de134fadad80e1e9101a8798/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:08.969972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688a7b8-0412-4f49-8201-29bddfefb22a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:08.970298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688a7b8-0412-4f49-8201-29bddfefb22a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:08.970577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688a7b8-0412-4f49-8201-29bddfefb22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:08.998800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-7688a7b8-0412-4f49-8201-29bddfefb22a None None] MFA Rules not processed for user `ec863386de134fadad80e1e9101a8798`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:08.999999 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-7688a7b8-0412-4f49-8201-29bddfefb22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:09.019260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e55c21a975a24d528285230988ade66c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.026561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e55c21a975a24d528285230988ade66c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.065087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1216/2424] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:09.070482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688a7b8-0412-4f49-8201-29bddfefb22a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.138860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688a7b8-0412-4f49-8201-29bddfefb22a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.149258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.149258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.149258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.149258 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.151019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.151019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.159311 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e55c21a975a24d528285230988ade66c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.159701 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e55c21a975a24d528285230988ade66c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.174708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e55c21a975a24d528285230988ade66c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.181357 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1209/2425] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:08 2022] DELETE /identity/v3/users/e55c21a975a24d528285230988ade66c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:09.271777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1217/2426] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:09 2022] POST /identity/v3/projects => generated 369 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:09.275725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.277518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:09.344774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09a3da-294f-449f-8e58-7f6cfcc5b754 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.405723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.406037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.406265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.406543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.407190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.409374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.521532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:09.534272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1210/2427] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:09 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:09.538044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.548986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358622a0-a8cc-4233-a2e0-3488f3debada admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.553719 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.554533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.555796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.555796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.555796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.557380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.568045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1218/2428] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:09.572964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.584181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34896-c488-4326-91e6-1a0b8d03054e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.589012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.589675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.589984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.590266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] PATH_INFO: `/v3/projects/6a5088203efc4d96b600716a9ba93ab3/users/84aab7920c7d44e4888fc5e76df57d2e/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.605814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5088203efc4d96b600716a9ba93ab3, user_id=84aab7920c7d44e4888fc5e76df57d2e, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.608486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.708982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:09.727962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.729713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:09.743998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdedeba-ccfc-4566-aa43-6c9b9904ae51 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.750532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.751256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.751605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.752282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] PATH_INFO: `/v3/projects/4c6d41041a31464db2da994ebc36b4d9` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.753853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6d41041a31464db2da994ebc36b4d9)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.755862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.817209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1211/2429] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:09 2022] PUT /identity/v3/projects/6a5088203efc4d96b600716a9ba93ab3/users/84aab7920c7d44e4888fc5e76df57d2e/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:09.821259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.892821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19198dc3-d1c8-4a6c-8fea-0d0874dd73c2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.898539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.899157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.899440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.899720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.900461 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.902838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.913568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1212/2430] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:09.917687 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:09.928416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e5367-4e4d-426c-a9a3-4df44e41a93f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:09.933487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c891fe-7c06-4df5-a355-c853eeaaa386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:09.934308 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c891fe-7c06-4df5-a355-c853eeaaa386 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.934661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c891fe-7c06-4df5-a355-c853eeaaa386 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.935013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c891fe-7c06-4df5-a355-c853eeaaa386 admin admin] PATH_INFO: `/v3/projects/6a5088203efc4d96b600716a9ba93ab3/users/84aab7920c7d44e4888fc5e76df57d2e/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.937205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c891fe-7c06-4df5-a355-c853eeaaa386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5088203efc4d96b600716a9ba93ab3, user_id=84aab7920c7d44e4888fc5e76df57d2e, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:09.940688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c891fe-7c06-4df5-a355-c853eeaaa386 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:09.953385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1213/2431] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:09 2022] PUT /identity/v3/projects/6a5088203efc4d96b600716a9ba93ab3/users/84aab7920c7d44e4888fc5e76df57d2e/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:09.958332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6d41041a31464db2da994ebc36b4d9', 'request_id': 'req-9e46dc36-2526-4968-a131-0bb6dd0d24c3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.965561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936b5a-5b5d-4af2-9c5c-ef2bd653168c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:09.966134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936b5a-5b5d-4af2-9c5c-ef2bd653168c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:09.966335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6d41041a31464db2da994ebc36b4d9', 'request_id': 'req-9e46dc36-2526-4968-a131-0bb6dd0d24c3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.966818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936b5a-5b5d-4af2-9c5c-ef2bd653168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:09.973988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6d41041a31464db2da994ebc36b4d9', 'request_id': 'req-9e46dc36-2526-4968-a131-0bb6dd0d24c3'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:09.996789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-1b936b5a-5b5d-4af2-9c5c-ef2bd653168c None None] MFA Rules not processed for user `84aab7920c7d44e4888fc5e76df57d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:09.998087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1b936b5a-5b5d-4af2-9c5c-ef2bd653168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.075659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1214/2432] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:10.075851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] The token cache is being invalidate because project 4c6d41041a31464db2da994ebc36b4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:10.075851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:10.076367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:10.077911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1219/2433] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:09 2022] DELETE /identity/v3/projects/4c6d41041a31464db2da994ebc36b4d9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:10.082463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82354183-b448-434f-919b-e2e8be892568 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.083340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82354183-b448-434f-919b-e2e8be892568 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.083620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82354183-b448-434f-919b-e2e8be892568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.083781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.085185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.103071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-82354183-b448-434f-919b-e2e8be892568 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:10.104286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-82354183-b448-434f-919b-e2e8be892568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.143887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46dc36-2526-4968-a131-0bb6dd0d24c3 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.167669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1215/2434] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:10.184108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0942-5a54-4ead-bd5c-c053d54e34ee None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.184538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0942-5a54-4ead-bd5c-c053d54e34ee None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.184936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0942-5a54-4ead-bd5c-c053d54e34ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.209622 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-acff0942-5a54-4ead-bd5c-c053d54e34ee None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:10.212522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-acff0942-5a54-4ead-bd5c-c053d54e34ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.215087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.215693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.215974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.216230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.218078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.232172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.234063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1216/2435] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:10.234297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.235185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0942-5a54-4ead-bd5c-c053d54e34ee None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.245888 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0942-5a54-4ead-bd5c-c053d54e34ee None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.247798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1220/2436] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:10.250897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.253528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.253528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.253528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.253528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:10.253528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.254936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.268503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1217/2437] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:10.279923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.292124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aaa18-020b-4b54-933e-18ca9193db6b service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.297334 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.297971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.298246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.298513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.299146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.301041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.492158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1221/2438] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/projects => generated 389 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:10.493544 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.500691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.504562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bef17e-3555-49cb-9bb1-223d33f1431b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.558111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfa749-abff-4148-8e82-3b08cc8e0fcf admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.561839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.562437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.562709 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.563081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.563309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.563954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.564223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.564493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.564867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:10.565137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.567223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.625058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.626656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.637213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1222/2439] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:10.723227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1218/2440] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/users => generated 295 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:10.724473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.735759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fea1c-b409-4e8c-aff1-16cadd78f30b service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.740395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.740991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.741263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.741543 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.742221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.743826 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.754257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1223/2441] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:10.758997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.769978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0195f-3813-4fa1-9a5a-c4b4dde14d38 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.776097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4f285-4e82-4992-bba5-2eceb1592de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.776723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f285-4e82-4992-bba5-2eceb1592de4 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.777010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f285-4e82-4992-bba5-2eceb1592de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.777321 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f285-4e82-4992-bba5-2eceb1592de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.778030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4f285-4e82-4992-bba5-2eceb1592de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.779629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4f285-4e82-4992-bba5-2eceb1592de4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.792111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1219/2442] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:10.796411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:10.807013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca0467-4ea7-4c94-b5d1-6b7d99740313 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:10.811656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:10.813335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.813335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.813335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] PATH_INFO: `/v3/projects/655bee338eb144e9b2d43429438f6309/users/2f59626792fc4169bd858d2d68cd81cd/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.828366 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] RBAC: Authorizing `identity:create_grant(project_id=655bee338eb144e9b2d43429438f6309, user_id=2f59626792fc4169bd858d2d68cd81cd, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:10.831176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:10.954074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1224/2443] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:10 2022] PUT /identity/v3/projects/655bee338eb144e9b2d43429438f6309/users/2f59626792fc4169bd858d2d68cd81cd/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:10.965361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3488c13-dc5b-4d4e-ad10-23bac09230e1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:10.965882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3488c13-dc5b-4d4e-ad10-23bac09230e1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:10.965980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3488c13-dc5b-4d4e-ad10-23bac09230e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:10.994667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a3488c13-dc5b-4d4e-ad10-23bac09230e1 None None] MFA Rules not processed for user `2f59626792fc4169bd858d2d68cd81cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:10.995887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a3488c13-dc5b-4d4e-ad10-23bac09230e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:11.062184 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1220/2444] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:11.067619 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.128234 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ce3b9-70e7-4088-9d9b-782ef67c500a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.134303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.135013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.135264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.135536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.136167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.138099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.286624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:11.294277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1225/2445] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:11 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:11.298694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3488c13-dc5b-4d4e-ad10-23bac09230e1 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.310806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3488c13-dc5b-4d4e-ad10-23bac09230e1 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.369932 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.370603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.370884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.371167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.371842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.373758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.557749 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1221/2446] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:11 2022] POST /identity/v3/users => generated 299 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:11.570259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.570259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bcbaa-42bb-42d1-a947-8af902d04d49 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.575082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.575699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.576005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.576284 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.576935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.578568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.590027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1226/2447] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:11.594354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.605213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b73d2-2be7-41ef-a7a7-fc66d59019df admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.609914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.610583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.610862 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.611136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] PATH_INFO: `/v3/projects/c175ba3c7ecc4e56a9556c6733031676/users/e5156b0e24e44ca19d03040c1ad39e6f/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.627111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c175ba3c7ecc4e56a9556c6733031676, user_id=e5156b0e24e44ca19d03040c1ad39e6f, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.630155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.757725 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1222/2448] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:11 2022] PUT /identity/v3/projects/c175ba3c7ecc4e56a9556c6733031676/users/e5156b0e24e44ca19d03040c1ad39e6f/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:11.758808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.776748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df30df-fc5f-4ce4-976d-97873a2ffedd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.828753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b527c-1e6e-46d8-849d-c09008fdc820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.829352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b527c-1e6e-46d8-849d-c09008fdc820 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.829649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b527c-1e6e-46d8-849d-c09008fdc820 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.829928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b527c-1e6e-46d8-849d-c09008fdc820 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.830559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b527c-1e6e-46d8-849d-c09008fdc820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.832147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b527c-1e6e-46d8-849d-c09008fdc820 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.842396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1227/2449] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:11.846500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:11.859490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122bcfe-833a-447f-96e0-453844471022 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:11.865832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:11.866552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.866879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.867158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] PATH_INFO: `/v3/projects/c175ba3c7ecc4e56a9556c6733031676/users/e5156b0e24e44ca19d03040c1ad39e6f/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.869185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c175ba3c7ecc4e56a9556c6733031676, user_id=e5156b0e24e44ca19d03040c1ad39e6f, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:11.872361 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:11.884964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1223/2450] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:11 2022] PUT /identity/v3/projects/c175ba3c7ecc4e56a9556c6733031676/users/e5156b0e24e44ca19d03040c1ad39e6f/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:11.898799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e62580-83ab-4d97-9c0a-82736d6b46d2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:11.899168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e62580-83ab-4d97-9c0a-82736d6b46d2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:11.899442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e62580-83ab-4d97-9c0a-82736d6b46d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:11.930238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-67e62580-83ab-4d97-9c0a-82736d6b46d2 None None] MFA Rules not processed for user `e5156b0e24e44ca19d03040c1ad39e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:11.931454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-67e62580-83ab-4d97-9c0a-82736d6b46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:12.000346 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1228/2451] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:12.011553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:12.069918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939561f-aa85-4720-bb87-4d073f4c5bd7 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:12.073560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63744354-f2d6-4f80-9398-64a1e399a7d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:12.074290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744354-f2d6-4f80-9398-64a1e399a7d0 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:12.074575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744354-f2d6-4f80-9398-64a1e399a7d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:12.074852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744354-f2d6-4f80-9398-64a1e399a7d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:12.085098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63744354-f2d6-4f80-9398-64a1e399a7d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:12.086667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63744354-f2d6-4f80-9398-64a1e399a7d0 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:12.096639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1224/2452] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:12.109304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:12.237429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e62580-83ab-4d97-9c0a-82736d6b46d2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:12.300734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e62580-83ab-4d97-9c0a-82736d6b46d2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:12.305376 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:12.306040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:12.306358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:12.306631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] PATH_INFO: `/v3/users/2f59626792fc4169bd858d2d68cd81cd` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:12.307828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f59626792fc4169bd858d2d68cd81cd)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:12.309795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:12.561571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f59626792fc4169bd858d2d68cd81cd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:12.568737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f59626792fc4169bd858d2d68cd81cd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:12.569415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f59626792fc4169bd858d2d68cd81cd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:12.570068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f59626792fc4169bd858d2d68cd81cd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:12.684849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f59626792fc4169bd858d2d68cd81cd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:12.703069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1229/2453] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:12 2022] DELETE /identity/v3/users/2f59626792fc4169bd858d2d68cd81cd => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:12.707557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:12.707839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:12.708132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:12.728591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:12.729842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:12.788412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1225/2454] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:12.799021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:12.800884 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:12.860198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c00f6-d152-49ff-811a-0e3169e99996 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:12.864271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:12.864907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:12.865185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:12.865453 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:12.875336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:12.877352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:12.888200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1230/2455] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:13.548000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:13.555569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:13.557048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:13.569979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da626b89-3a7e-4f51-b114-4d9ba8685eec None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:13.575295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:13.575937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:13.576216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:13.576490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] PATH_INFO: `/v3/projects/655bee338eb144e9b2d43429438f6309` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:13.577624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] RBAC: Authorizing `identity:delete_project(project_id=655bee338eb144e9b2d43429438f6309)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:13.579571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:13.746954 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '655bee338eb144e9b2d43429438f6309', 'request_id': 'req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.753628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '655bee338eb144e9b2d43429438f6309', 'request_id': 'req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.851196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:13.890474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '655bee338eb144e9b2d43429438f6309', 'request_id': 'req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.904879 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] The token cache is being invalidate because project 655bee338eb144e9b2d43429438f6309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:13.905178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.906125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:13.908118 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1226/2456] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:13 2022] DELETE /identity/v3/projects/655bee338eb144e9b2d43429438f6309 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:13.911921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:13.923528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db82b-b7c4-49e5-afdd-9ed6ccd193cd service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:13.991830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:14.003828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:14.003828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:14.003828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] PATH_INFO: `/v3/users/e5156b0e24e44ca19d03040c1ad39e6f` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:14.003828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5156b0e24e44ca19d03040c1ad39e6f)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:14.003828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:14.258928 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5156b0e24e44ca19d03040c1ad39e6f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.265892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5156b0e24e44ca19d03040c1ad39e6f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.266559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5156b0e24e44ca19d03040c1ad39e6f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.267156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5156b0e24e44ca19d03040c1ad39e6f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.364986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5156b0e24e44ca19d03040c1ad39e6f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:14.385833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1231/2457] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:13 2022] DELETE /identity/v3/users/e5156b0e24e44ca19d03040c1ad39e6f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:14.754028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:14.855066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:14.855066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:14.864689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2dfbd-57b3-4c5c-8a2a-cf90f200323c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:14.903021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:14.903021 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:14.935242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:14.935998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:14.936349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:14.936695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] PATH_INFO: `/v3/projects/c175ba3c7ecc4e56a9556c6733031676` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:14.937886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c175ba3c7ecc4e56a9556c6733031676)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:14.939795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:14.958431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31efa09-038b-421e-8fe9-de9750e4fb4e admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:14.972059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83514fa2-98de-4326-b9cf-d81ab755b90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:14.972059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83514fa2-98de-4326-b9cf-d81ab755b90e service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:14.972059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83514fa2-98de-4326-b9cf-d81ab755b90e service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:14.972059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83514fa2-98de-4326-b9cf-d81ab755b90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:14.972423 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-83514fa2-98de-4326-b9cf-d81ab755b90e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:14.978239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83514fa2-98de-4326-b9cf-d81ab755b90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:14.979683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83514fa2-98de-4326-b9cf-d81ab755b90e service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:14.988275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:14.990268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1232/2458] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:15.127510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c175ba3c7ecc4e56a9556c6733031676', 'request_id': 'req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:15.134416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c175ba3c7ecc4e56a9556c6733031676', 'request_id': 'req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:15.267416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c175ba3c7ecc4e56a9556c6733031676', 'request_id': 'req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:15.287569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] The token cache is being invalidate because project c175ba3c7ecc4e56a9556c6733031676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:15.288123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:15.288880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-f365f666-1b0f-47ec-b9ef-fa6b60b6f5bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:15.291431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1227/2459] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:14 2022] DELETE /identity/v3/projects/c175ba3c7ecc4e56a9556c6733031676 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:15.296455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7ef0-5086-4928-837a-db7697d7a3f7 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:15.296935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7ef0-5086-4928-837a-db7697d7a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:15.297216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7ef0-5086-4928-837a-db7697d7a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:15.325339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-005b7ef0-5086-4928-837a-db7697d7a3f7 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:15.325339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-005b7ef0-5086-4928-837a-db7697d7a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:15.382736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1233/2460] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:15.392111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec2d0-c4c6-40d2-9f01-e980dce56ecb None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:15.392111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec2d0-c4c6-40d2-9f01-e980dce56ecb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:15.392381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ec2d0-c4c6-40d2-9f01-e980dce56ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:15.412438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a84ec2d0-c4c6-40d2-9f01-e980dce56ecb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:15.413649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a84ec2d0-c4c6-40d2-9f01-e980dce56ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:15.422444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1228/2461] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:15.426716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b7ef0-5086-4928-837a-db7697d7a3f7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:15.438017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b7ef0-5086-4928-837a-db7697d7a3f7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:15.442887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:15.447652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:15.447652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:15.447652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:15.447652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:15.447652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:15.447652 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:15.464886 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1234/2462] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:15.466895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ec2d0-c4c6-40d2-9f01-e980dce56ecb None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:15.479126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ec2d0-c4c6-40d2-9f01-e980dce56ecb None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:15.486121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:15.486121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:15.486121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:15.486121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:15.486660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:15.488032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:15.726685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1229/2463] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:15 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:15.731565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:15.741822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c914b-ae9f-4de5-adf5-1733771061a2 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:15.802416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:15.803482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:15.803974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:15.804534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:15.805693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:15.809457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.019855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1235/2464] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:15 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:16.021023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.032303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26774ec-9d27-46e5-8aa0-d589531e9983 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.037093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.037710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.038066 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.038342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.039030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.040697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.051915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1230/2465] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:16.060130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.072217 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f43a-499b-48fe-80f9-8f19050f22b8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.077136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a7f58-ef22-41e3-a2d3-f19d8b5187c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.077919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a7f58-ef22-41e3-a2d3-f19d8b5187c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.078252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a7f58-ef22-41e3-a2d3-f19d8b5187c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.078536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a7f58-ef22-41e3-a2d3-f19d8b5187c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.079190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a7f58-ef22-41e3-a2d3-f19d8b5187c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.080831 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a7f58-ef22-41e3-a2d3-f19d8b5187c5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.092499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1236/2466] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:16.096848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.107245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126bd2d-658b-41bf-aee7-a63cc2edd7d2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.111974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.112627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.112912 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.113188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] PATH_INFO: `/v3/projects/dce444ba506a4dfd89826d0192986483/users/389e9046d5cd4596bb6fcb495cf31c9d/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.128464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce444ba506a4dfd89826d0192986483, user_id=389e9046d5cd4596bb6fcb495cf31c9d, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.131192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.290648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1231/2467] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:16 2022] PUT /identity/v3/projects/dce444ba506a4dfd89826d0192986483/users/389e9046d5cd4596bb6fcb495cf31c9d/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:16.292175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738599ca-8fd4-458f-a846-af0bfbfc1263 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.292485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738599ca-8fd4-458f-a846-af0bfbfc1263 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.292758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738599ca-8fd4-458f-a846-af0bfbfc1263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.307843 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:16.326987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-738599ca-8fd4-458f-a846-af0bfbfc1263 None None] MFA Rules not processed for user `389e9046d5cd4596bb6fcb495cf31c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:16.328303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-738599ca-8fd4-458f-a846-af0bfbfc1263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.445009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1237/2468] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:16.449291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.525422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f15654-f686-498f-8780-03696146ab13 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.530574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.531317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.531625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.532055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.532712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.534756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.700993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1232/2469] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/projects => generated 377 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:16.702246 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738599ca-8fd4-458f-a846-af0bfbfc1263 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.714786 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738599ca-8fd4-458f-a846-af0bfbfc1263 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.778029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.778683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.778968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.779243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.779889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.782028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.816953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:16.926340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.931992 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:16.944209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1238/2470] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:16 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:16.956915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:16.960994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84e838-fa00-4e9a-a3ca-cf593d4a7b24 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.969964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.970751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.971023 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.971298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] PATH_INFO: `/v3/users/ec863386de134fadad80e1e9101a8798` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.971736 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7c40-6ce8-410f-820d-ea3fe1eed7a4 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:16.972531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec863386de134fadad80e1e9101a8798)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.974975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:16.977911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:16.979954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:16.980540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:16.981019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:16.981956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:16.984967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.001541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1239/2471] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:17.008529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.022007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8a9c5-2734-4a35-91c2-67da8c986ec3 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.030568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.031352 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.040087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.040087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.040087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.040087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.049225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1240/2472] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:17.054160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.067038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e2c8d-a84b-4e00-98f7-95f2cacdd0fd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.074208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c1131-7944-4d35-b5da-a2f75e5ddfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.075136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c1131-7944-4d35-b5da-a2f75e5ddfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.088047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c1131-7944-4d35-b5da-a2f75e5ddfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.088047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c1131-7944-4d35-b5da-a2f75e5ddfd8 admin admin] PATH_INFO: `/v3/projects/27259e770cca4c1bb4fdd34aad21cdec/users/8b9d2ee798f64f498be1b1140e365e2d/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.098946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c1131-7944-4d35-b5da-a2f75e5ddfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27259e770cca4c1bb4fdd34aad21cdec, user_id=8b9d2ee798f64f498be1b1140e365e2d, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.101688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c1131-7944-4d35-b5da-a2f75e5ddfd8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.204421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1241/2473] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:17 2022] PUT /identity/v3/projects/27259e770cca4c1bb4fdd34aad21cdec/users/8b9d2ee798f64f498be1b1140e365e2d/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:17.205199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05401053-1a2f-4744-bdf2-aa4fefd61d2e None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.205526 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05401053-1a2f-4744-bdf2-aa4fefd61d2e None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.205747 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05401053-1a2f-4744-bdf2-aa4fefd61d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.239740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-05401053-1a2f-4744-bdf2-aa4fefd61d2e None None] MFA Rules not processed for user `8b9d2ee798f64f498be1b1140e365e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:17.262966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-05401053-1a2f-4744-bdf2-aa4fefd61d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:17.344298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1242/2474] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:17.348979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05401053-1a2f-4744-bdf2-aa4fefd61d2e None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.352766 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec863386de134fadad80e1e9101a8798'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:17.362568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec863386de134fadad80e1e9101a8798'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:17.418613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05401053-1a2f-4744-bdf2-aa4fefd61d2e None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.424957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56bde-3fc5-4a49-b275-f638a7d241f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.425654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56bde-3fc5-4a49-b275-f638a7d241f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.425971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56bde-3fc5-4a49-b275-f638a7d241f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.426245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56bde-3fc5-4a49-b275-f638a7d241f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.426905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56bde-3fc5-4a49-b275-f638a7d241f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.429061 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56bde-3fc5-4a49-b275-f638a7d241f0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.554807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec863386de134fadad80e1e9101a8798'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:17.557381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec863386de134fadad80e1e9101a8798'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:17.574136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec863386de134fadad80e1e9101a8798'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:17.583841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1233/2475] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:16 2022] DELETE /identity/v3/users/ec863386de134fadad80e1e9101a8798 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:17.584478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1243/2476] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:17 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:17.588839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.590527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.590880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.591531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.595794 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:17.620961 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:17.623478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:17.699520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84a059-ac39-4c54-a0fb-4251dcc13edf admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.705169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.705878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.706270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.707509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.707509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.709242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.740540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1244/2477] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:17.750979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.752848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:17.788694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1234/2478] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:17 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:17.793507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.804466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306362-7daa-4f5e-b5c8-fc3019bc16c8 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.809546 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.810272 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.810555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.810833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.811501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.813121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.826958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf7d78-9531-490f-913c-c43d5f8795b2 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.833252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1235/2479] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:17.833503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.833503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.833503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.834271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.835221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.846101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.847054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ea440-5d81-4164-a9cc-bc68565dc1ad admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:17.848278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.852858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:17.853568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:17.853878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:17.854152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] PATH_INFO: `/v3/projects/f3ed2bef54ce4d98bf23ae15e0a7d2d7/users/748e9238ddd946ac96482cfa201182e4/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:17.861948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1245/2480] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:17.873264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ed2bef54ce4d98bf23ae15e0a7d2d7, user_id=748e9238ddd946ac96482cfa201182e4, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:17.877003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:17.978636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1236/2481] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:17 2022] PUT /identity/v3/projects/f3ed2bef54ce4d98bf23ae15e0a7d2d7/users/748e9238ddd946ac96482cfa201182e4/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:17.983076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:17.995132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed7601-da9a-49f3-b325-00047454e9e2 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.076855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ff0cb-28af-4d33-84cd-40a54b59a847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.077973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ff0cb-28af-4d33-84cd-40a54b59a847 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.078452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ff0cb-28af-4d33-84cd-40a54b59a847 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.078924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ff0cb-28af-4d33-84cd-40a54b59a847 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.079987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ff0cb-28af-4d33-84cd-40a54b59a847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.082130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ff0cb-28af-4d33-84cd-40a54b59a847 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.086730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:18.110857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1246/2482] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:18.112340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.123605 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9ab33-1700-4eae-95a5-96c65016e118 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.128614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.131469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.131469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.131469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] PATH_INFO: `/v3/projects/f3ed2bef54ce4d98bf23ae15e0a7d2d7/users/748e9238ddd946ac96482cfa201182e4/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.132363 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ed2bef54ce4d98bf23ae15e0a7d2d7, user_id=748e9238ddd946ac96482cfa201182e4, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.151556 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.165386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1237/2483] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:18 2022] PUT /identity/v3/projects/f3ed2bef54ce4d98bf23ae15e0a7d2d7/users/748e9238ddd946ac96482cfa201182e4/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:18.179495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.179495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.180962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.226445 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] MFA Rules not processed for user `748e9238ddd946ac96482cfa201182e4`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:18.227980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.352363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1247/2484] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:18.362062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.369511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.458081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376aae2-cdea-4365-9af4-0412cd222b1f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.461670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.462350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.462627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.462898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.484487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.484806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.484912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.485021 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.494484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1238/2485] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:18.593039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:18.597603 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4156d-2fb5-4bd3-85b0-53ad7af54321 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.603635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.604361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.604645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.604924 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] PATH_INFO: `/v3/projects/85e47a815aea43da82c2a2f518511e7a` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.606252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e47a815aea43da82c2a2f518511e7a)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.608257 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.656458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:18.657947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.671230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab1b9-5d9f-4b58-80bf-0e0a78904076 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:18.675576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:18.676244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:18.676522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:18.676880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:18.678772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:18.745541 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:18.746922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:18.757540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1239/2486] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:18.826530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e47a815aea43da82c2a2f518511e7a', 'request_id': 'req-3a573e2c-5323-4af1-a75e-299aa286ac4d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.833958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e47a815aea43da82c2a2f518511e7a', 'request_id': 'req-3a573e2c-5323-4af1-a75e-299aa286ac4d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.841647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e47a815aea43da82c2a2f518511e7a', 'request_id': 'req-3a573e2c-5323-4af1-a75e-299aa286ac4d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.975491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] The token cache is being invalidate because project 85e47a815aea43da82c2a2f518511e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:18.977564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.981194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:18.983676 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1248/2487] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:18 2022] DELETE /identity/v3/projects/85e47a815aea43da82c2a2f518511e7a => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:18.987873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:19.012975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502e39-abe9-4337-9c34-ba42fbcaf877 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:19.103974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:19.106711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:19.106711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:19.106711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] PATH_INFO: `/v3/users/84aab7920c7d44e4888fc5e76df57d2e` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:19.106711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] RBAC: Authorizing `identity:delete_user(user_id=84aab7920c7d44e4888fc5e76df57d2e)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:19.109005 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:19.501837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84aab7920c7d44e4888fc5e76df57d2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.515320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84aab7920c7d44e4888fc5e76df57d2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.607740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84aab7920c7d44e4888fc5e76df57d2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.608570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84aab7920c7d44e4888fc5e76df57d2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.623773 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84aab7920c7d44e4888fc5e76df57d2e'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:19.630820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1240/2488] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:18 2022] DELETE /identity/v3/users/84aab7920c7d44e4888fc5e76df57d2e => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:20.100120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:20.103171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:20.128241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a573e2c-5323-4af1-a75e-299aa286ac4d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:20.262881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:20.263799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:20.264249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:20.264586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] PATH_INFO: `/v3/users/c0afa15991f14e0ca0d3701d06a89049` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:20.265894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0afa15991f14e0ca0d3701d06a89049)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:20.268117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:20.407654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:20.409166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:20.424949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd0124-8ede-4729-adb9-7388108f19da admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:20.432040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:20.433618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:20.434196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:20.434690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] PATH_INFO: `/v3/projects/6a5088203efc4d96b600716a9ba93ab3` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:20.436281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5088203efc4d96b600716a9ba93ab3)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:20.439002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:20.671911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0afa15991f14e0ca0d3701d06a89049'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.676289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5088203efc4d96b600716a9ba93ab3', 'request_id': 'req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.682260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5088203efc4d96b600716a9ba93ab3', 'request_id': 'req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.684693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0afa15991f14e0ca0d3701d06a89049'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.685552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0afa15991f14e0ca0d3701d06a89049'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.686194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0afa15991f14e0ca0d3701d06a89049'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.919178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5088203efc4d96b600716a9ba93ab3', 'request_id': 'req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.935925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] The token cache is being invalidate because project 6a5088203efc4d96b600716a9ba93ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:20.936849 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.937464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1e243158-d15a-4ab4-b84d-ce0e63f6bbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:20.939920 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1241/2489] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:20 2022] DELETE /identity/v3/projects/6a5088203efc4d96b600716a9ba93ab3 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:20.944645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02641af-073c-4e2a-b5b9-ecaba363f920 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:20.944938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02641af-073c-4e2a-b5b9-ecaba363f920 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:20.945217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02641af-073c-4e2a-b5b9-ecaba363f920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:20.965908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c02641af-073c-4e2a-b5b9-ecaba363f920 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:20.967167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c02641af-073c-4e2a-b5b9-ecaba363f920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.034341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:21.040038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0afa15991f14e0ca0d3701d06a89049'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.045121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1242/2490] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:21.054922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668e41e-1e0c-4904-9b09-b48be696be29 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.055707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668e41e-1e0c-4904-9b09-b48be696be29 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.055980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668e41e-1e0c-4904-9b09-b48be696be29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.076836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1249/2491] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:20 2022] DELETE /identity/v3/users/c0afa15991f14e0ca0d3701d06a89049 => generated 0 bytes in 978 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:21.084875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9668e41e-1e0c-4904-9b09-b48be696be29 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:21.086651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9668e41e-1e0c-4904-9b09-b48be696be29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.170597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1243/2492] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:21.175224 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.182002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.198282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ddce9-8ab8-4071-9592-bb4c2538c401 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.204912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.205546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.205855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.206169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.206983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:21.207901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.209600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.222825 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1250/2493] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:08:21.227484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668e41e-1e0c-4904-9b09-b48be696be29 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.241364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668e41e-1e0c-4904-9b09-b48be696be29 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.247349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.247981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.248286 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.248554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.249196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.251146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.446542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1244/2494] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:21 2022] POST /identity/v3/projects => generated 401 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:21.447545 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.459677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded3e05-be70-4e1c-992c-88d5a0443d14 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.483225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.485798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.540770 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.548413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.548413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.548413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.548413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.548413 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.577707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308a185-ff13-47a9-9987-bd1645a4b9ff admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.582524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.583157 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.583437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.583712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] PATH_INFO: `/v3/projects/c19855b809cc43b2925dc3001284baaa` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.584844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19855b809cc43b2925dc3001284baaa)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.586890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.656666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1251/2495] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:21 2022] POST /identity/v3/users => generated 301 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:21.661067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.672095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d2aa-4b70-4b30-8b4e-3d8e89407b48 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.676707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.677297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.677597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.677901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.678581 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.680697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.691323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:21.694279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1252/2496] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:21.703795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.710683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc58df-3601-4b4b-a5cd-b6204a832e25 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.716316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.716933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.717212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.717485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.718177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.719783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.738463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1253/2497] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:21.738463 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:21.748519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3525988-4786-4100-bf79-e28f3cf24662 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:21.755810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d3035-f000-4619-883f-e74ceeb9b1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:21.756518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d3035-f000-4619-883f-e74ceeb9b1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.756824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d3035-f000-4619-883f-e74ceeb9b1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.757082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d3035-f000-4619-883f-e74ceeb9b1dd admin admin] PATH_INFO: `/v3/projects/1048c52856214a36b0baa3e60ec69c84/users/a4c293f32c68463990ff4912a91bcd87/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.777555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d3035-f000-4619-883f-e74ceeb9b1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1048c52856214a36b0baa3e60ec69c84, user_id=a4c293f32c68463990ff4912a91bcd87, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:21.779916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19855b809cc43b2925dc3001284baaa', 'request_id': 'req-73829f9b-db64-46dc-b4d9-0025d903d8e6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.780401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d3035-f000-4619-883f-e74ceeb9b1dd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:21.785801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19855b809cc43b2925dc3001284baaa', 'request_id': 'req-73829f9b-db64-46dc-b4d9-0025d903d8e6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.890741 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19855b809cc43b2925dc3001284baaa', 'request_id': 'req-73829f9b-db64-46dc-b4d9-0025d903d8e6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.897144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1254/2498] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:21 2022] PUT /identity/v3/projects/1048c52856214a36b0baa3e60ec69c84/users/a4c293f32c68463990ff4912a91bcd87/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:21.903875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] The token cache is being invalidate because project c19855b809cc43b2925dc3001284baaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:21.904223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.904939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-73829f9b-db64-46dc-b4d9-0025d903d8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:21.906659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1245/2499] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:21 2022] DELETE /identity/v3/projects/c19855b809cc43b2925dc3001284baaa => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:21.907972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.908277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.908623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.911674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf389-3300-47ce-accf-4e88cd3b3100 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:21.911937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf389-3300-47ce-accf-4e88cd3b3100 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:21.912219 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf389-3300-47ce-accf-4e88cd3b3100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:21.933514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0d9bf389-3300-47ce-accf-4e88cd3b3100 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:21.934824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0d9bf389-3300-47ce-accf-4e88cd3b3100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:21.953971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] MFA Rules not processed for user `a4c293f32c68463990ff4912a91bcd87`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:21.955367 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.013599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1246/2500] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:22.023280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b966c5-b6f6-4c68-a54d-c397316d952c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.024204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b966c5-b6f6-4c68-a54d-c397316d952c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.024668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b966c5-b6f6-4c68-a54d-c397316d952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.052864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-c2b966c5-b6f6-4c68-a54d-c397316d952c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:22.054135 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c2b966c5-b6f6-4c68-a54d-c397316d952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.062865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1247/2501] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:22.066546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1255/2502] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:21 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:22.070114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b966c5-b6f6-4c68-a54d-c397316d952c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.071414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.072779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.076752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b966c5-b6f6-4c68-a54d-c397316d952c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.089287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.089287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.089287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.089287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.089716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7f21-d743-4ca0-80f8-bed837f84f37 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.089832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:22.089832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.090422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.094893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.095501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.095777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.096048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.096711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.098918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.103189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1248/2503] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:08:22.112653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.124017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461a5de-135f-45b3-b9d4-23b500d6bbe7 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.129591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.130449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.130727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.131440 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.132082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.134689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.200432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1256/2504] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/projects => generated 399 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:22.204429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.269910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1249/2505] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:22.271253 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b531440-323a-4af9-adf3-61de54643e9c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.278940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.336715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fe521-cdb9-42f4-96ab-25896a879237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.337389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe521-cdb9-42f4-96ab-25896a879237 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.337720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe521-cdb9-42f4-96ab-25896a879237 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.338040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe521-cdb9-42f4-96ab-25896a879237 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.338678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fe521-cdb9-42f4-96ab-25896a879237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.340677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fe521-cdb9-42f4-96ab-25896a879237 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.345025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44d9db-e4cf-4a0a-8f0a-60b4e9876ffb admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.350070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.350524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.350805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.351081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.352345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.354197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.451598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1257/2506] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/users => generated 306 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:22.452966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edd3fa-ea84-4ada-bd3a-1c0a12a34fab None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.453259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edd3fa-ea84-4ada-bd3a-1c0a12a34fab None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.453538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edd3fa-ea84-4ada-bd3a-1c0a12a34fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.469970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1250/2507] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:22.470694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.477877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c5edd3fa-ea84-4ada-bd3a-1c0a12a34fab None None] MFA Rules not processed for user `748b8b4771ac498a88fdd98c8a4914ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:22.479466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c5edd3fa-ea84-4ada-bd3a-1c0a12a34fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.482469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443a347-b66f-49e4-9100-8a5dd6e04931 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.487893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.488504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.488783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.489105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.489705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.491332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.502553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1251/2508] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:22.503498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.516175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465a70a-cb2a-4279-aea5-b08c08986007 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.520980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.527383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.527383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.527383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.527383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.527383 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.535426 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1252/2509] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:22.536331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.539465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1258/2510] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:22.543074 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edd3fa-ea84-4ada-bd3a-1c0a12a34fab None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.547428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d4a98-cf9b-4e3e-b679-aeb1eef45006 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.552958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.553616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.553922 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.554197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] PATH_INFO: `/v3/projects/c69a4801b3be4766ab28ea7853e15056/users/d96f911def6c412287a6b1842616b1c8/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.556797 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edd3fa-ea84-4ada-bd3a-1c0a12a34fab None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.563407 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.564424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.564945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.565426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.566100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.568012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.571578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69a4801b3be4766ab28ea7853e15056, user_id=d96f911def6c412287a6b1842616b1c8, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.576675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.579783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1259/2511] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:22.583870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.594779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db3f4f-c5c9-407c-b92c-f641f8dd35da admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.599796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf9937-84de-4493-8507-4816d4f9aae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.600136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf9937-84de-4493-8507-4816d4f9aae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.600410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf9937-84de-4493-8507-4816d4f9aae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.600679 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf9937-84de-4493-8507-4816d4f9aae2 admin admin] PATH_INFO: `/v3/projects/a2a8b05bca8d4a30ad3286d5a10ad923/users/13f2f26e9daf416ba46371b5948ab743/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.616656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf9937-84de-4493-8507-4816d4f9aae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a8b05bca8d4a30ad3286d5a10ad923, user_id=13f2f26e9daf416ba46371b5948ab743, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.620033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf9937-84de-4493-8507-4816d4f9aae2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.708126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1253/2512] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:22 2022] PUT /identity/v3/projects/c69a4801b3be4766ab28ea7853e15056/users/d96f911def6c412287a6b1842616b1c8/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:22.712255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.743721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1260/2513] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:22 2022] PUT /identity/v3/projects/a2a8b05bca8d4a30ad3286d5a10ad923/users/13f2f26e9daf416ba46371b5948ab743/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:22.758518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.758805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.759166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.781463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed59404-1b73-4d2e-aaca-1b88c5a2f231 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.789913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] MFA Rules not processed for user `13f2f26e9daf416ba46371b5948ab743`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:22.791161 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.854231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.854231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.854231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.854231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.854729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.854729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.867626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1254/2514] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:22.868341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1261/2515] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:22.871806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.883635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:22.885525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.888787 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bd23d-5a48-46a4-87c6-c6c61e7fe29c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:22.894579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3812ea3-0b67-4c57-8910-24df14188ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:22.895254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3812ea3-0b67-4c57-8910-24df14188ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.895537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3812ea3-0b67-4c57-8910-24df14188ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.895816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3812ea3-0b67-4c57-8910-24df14188ec7 admin admin] PATH_INFO: `/v3/projects/c69a4801b3be4766ab28ea7853e15056/users/d96f911def6c412287a6b1842616b1c8/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.897940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3812ea3-0b67-4c57-8910-24df14188ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69a4801b3be4766ab28ea7853e15056, user_id=d96f911def6c412287a6b1842616b1c8, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:22.901153 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3812ea3-0b67-4c57-8910-24df14188ec7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:22.914370 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1255/2516] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:22 2022] PUT /identity/v3/projects/c69a4801b3be4766ab28ea7853e15056/users/d96f911def6c412287a6b1842616b1c8/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:22.925709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:22.926105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:22.926459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:22.960231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] MFA Rules not processed for user `d96f911def6c412287a6b1842616b1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:22.961415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:22.970935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9539a6-c344-4ba1-be22-a21f720869ef None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.045938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1256/2517] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:23.055259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.055259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.055259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.055259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.057470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:23.061409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:23.118437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.120103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.130381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d0c4-092d-49a0-8de8-498f3bba25a6 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:23.130527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1262/2518] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:23.136360 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:23.136984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:23.137260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:23.137529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:23.147501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:23.149474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:23.159708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1257/2519] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:24.033907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:24.035578 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:24.055163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49fe61-070e-4aa2-9c6e-66e9fd772a5b service nova] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:24.055163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092290f1-5d1c-4d7d-8dc8-0029f7db52dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:24.055163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092290f1-5d1c-4d7d-8dc8-0029f7db52dc service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.055460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092290f1-5d1c-4d7d-8dc8-0029f7db52dc service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.055547 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092290f1-5d1c-4d7d-8dc8-0029f7db52dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.119525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092290f1-5d1c-4d7d-8dc8-0029f7db52dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:24.121106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092290f1-5d1c-4d7d-8dc8-0029f7db52dc service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:24.131225 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1263/2520] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:24 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:24.343747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:24.345213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:24.383525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:24.414518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb294b-3ef2-4702-b699-1aa5a30a3672 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:24.419166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:24.419821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.420182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.420414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] PATH_INFO: `/v3/users/389e9046d5cd4596bb6fcb495cf31c9d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.421740 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=389e9046d5cd4596bb6fcb495cf31c9d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:24.423665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:24.840729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389e9046d5cd4596bb6fcb495cf31c9d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.848274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389e9046d5cd4596bb6fcb495cf31c9d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.942069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389e9046d5cd4596bb6fcb495cf31c9d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.942855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389e9046d5cd4596bb6fcb495cf31c9d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.958607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389e9046d5cd4596bb6fcb495cf31c9d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:24.967887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1258/2521] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:24 2022] DELETE /identity/v3/users/389e9046d5cd4596bb6fcb495cf31c9d => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:24.972575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:24.972859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:24.973131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:24.993734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:24.995243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.068363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1264/2522] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:25.078438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.079875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.140092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:25.145389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d3652-d638-4953-813a-5a91a6834aa2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.149046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.149652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.149960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.150235 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.159643 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.161060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:25.171480 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1259/2523] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:25.718906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:25.718906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:25.729530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac662e0-914b-4e20-be7c-2a54611f1663 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:25.734655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:25.735318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:25.735599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:25.735946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] PATH_INFO: `/v3/projects/dce444ba506a4dfd89826d0192986483` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:25.737118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce444ba506a4dfd89826d0192986483)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:25.739117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:25.988830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce444ba506a4dfd89826d0192986483', 'request_id': 'req-9df8feb2-7de0-470c-9c55-b6239ca0067c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:25.995487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce444ba506a4dfd89826d0192986483', 'request_id': 'req-9df8feb2-7de0-470c-9c55-b6239ca0067c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.001120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce444ba506a4dfd89826d0192986483', 'request_id': 'req-9df8feb2-7de0-470c-9c55-b6239ca0067c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.220739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] The token cache is being invalidate because project dce444ba506a4dfd89826d0192986483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:26.220739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.221476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.224207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1265/2524] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:25 2022] DELETE /identity/v3/projects/dce444ba506a4dfd89826d0192986483 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:26.231015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:26.247530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46932a7-4570-4190-ada1-9216cdf165f9 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:26.339641 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:26.344824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:26.344824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:26.344824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] PATH_INFO: `/v3/users/8b9d2ee798f64f498be1b1140e365e2d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:26.344824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9d2ee798f64f498be1b1140e365e2d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:26.344824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:26.649166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9d2ee798f64f498be1b1140e365e2d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.657087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9d2ee798f64f498be1b1140e365e2d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.741710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9d2ee798f64f498be1b1140e365e2d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.742870 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9d2ee798f64f498be1b1140e365e2d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.757513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9d2ee798f64f498be1b1140e365e2d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:26.764420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1260/2525] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:26 2022] DELETE /identity/v3/users/8b9d2ee798f64f498be1b1140e365e2d => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:27.376807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:27.376807 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:27.397146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8feb2-7de0-470c-9c55-b6239ca0067c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:27.459240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:27.459948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:27.460256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:27.460532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] PATH_INFO: `/v3/projects/27259e770cca4c1bb4fdd34aad21cdec` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:27.462306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=27259e770cca4c1bb4fdd34aad21cdec)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:27.464415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:27.619592 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27259e770cca4c1bb4fdd34aad21cdec', 'request_id': 'req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:27.626136 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27259e770cca4c1bb4fdd34aad21cdec', 'request_id': 'req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:27.632371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27259e770cca4c1bb4fdd34aad21cdec', 'request_id': 'req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:27.731009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] The token cache is being invalidate because project 27259e770cca4c1bb4fdd34aad21cdec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:27.731299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:27.731957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:27.733495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1266/2526] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:27 2022] DELETE /identity/v3/projects/27259e770cca4c1bb4fdd34aad21cdec => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:27.737764 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:27.739300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:27.753087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08625914-37ee-4217-8220-101f0abeb419 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:27.781726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:27.814380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:27.815119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:27.815385 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:27.815665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] PATH_INFO: `/v3/users/748e9238ddd946ac96482cfa201182e4` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:27.816833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=748e9238ddd946ac96482cfa201182e4)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:27.818820 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:28.016860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:28.112400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748e9238ddd946ac96482cfa201182e4'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.120154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748e9238ddd946ac96482cfa201182e4'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.239530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748e9238ddd946ac96482cfa201182e4'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.240223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748e9238ddd946ac96482cfa201182e4'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.259040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-2d21099b-fa60-463a-8917-844fc32fca90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748e9238ddd946ac96482cfa201182e4'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.266281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1261/2527] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:27 2022] DELETE /identity/v3/users/748e9238ddd946ac96482cfa201182e4 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:28.774943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:28.781987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:28.802414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca3ad2-a77a-4246-a0fd-f4c7f25f53bd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:28.868321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:28.868982 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:28.869263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:28.869540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] PATH_INFO: `/v3/projects/f3ed2bef54ce4d98bf23ae15e0a7d2d7` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:28.870742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3ed2bef54ce4d98bf23ae15e0a7d2d7)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:28.872753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:28.925312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:28.982651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3ed2bef54ce4d98bf23ae15e0a7d2d7', 'request_id': 'req-e1bfce80-11a3-4687-932f-24c30a7276aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.988919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3ed2bef54ce4d98bf23ae15e0a7d2d7', 'request_id': 'req-e1bfce80-11a3-4687-932f-24c30a7276aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:28.994404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3ed2bef54ce4d98bf23ae15e0a7d2d7', 'request_id': 'req-e1bfce80-11a3-4687-932f-24c30a7276aa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.095965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] The token cache is being invalidate because project f3ed2bef54ce4d98bf23ae15e0a7d2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:29.099316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.099316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e1bfce80-11a3-4687-932f-24c30a7276aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:29.099316 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1267/2528] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:28 2022] DELETE /identity/v3/projects/f3ed2bef54ce4d98bf23ae15e0a7d2d7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:29.105310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca37dc-cd51-4a5c-8248-1a92c30b5e61 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.105855 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca37dc-cd51-4a5c-8248-1a92c30b5e61 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.105955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca37dc-cd51-4a5c-8248-1a92c30b5e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.126046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ecca37dc-cd51-4a5c-8248-1a92c30b5e61 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:29.133925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ecca37dc-cd51-4a5c-8248-1a92c30b5e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:29.189726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1262/2529] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:29.199383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6af6b1-666b-48fe-a1e1-ecc146ebed24 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.199667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6af6b1-666b-48fe-a1e1-ecc146ebed24 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.199942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6af6b1-666b-48fe-a1e1-ecc146ebed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.221385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-fd6af6b1-666b-48fe-a1e1-ecc146ebed24 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:29.238597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fd6af6b1-666b-48fe-a1e1-ecc146ebed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:29.238597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1268/2530] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:29.241674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca37dc-cd51-4a5c-8248-1a92c30b5e61 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.252871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca37dc-cd51-4a5c-8248-1a92c30b5e61 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.258588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.259239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.260052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.260052 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.260917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:29.260917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.262492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.276273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1263/2531] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:29.280951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6af6b1-666b-48fe-a1e1-ecc146ebed24 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.291997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6af6b1-666b-48fe-a1e1-ecc146ebed24 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.298400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.299029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.299307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.299623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.300311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.302506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.330373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:29.598755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1269/2532] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:29 2022] POST /identity/v3/projects => generated 383 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:29.603483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.613752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e71091-ada0-4cb4-8389-e27a7e73925b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.676080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.676686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.676955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.677222 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.677899 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.680282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.736500 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:29.788316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1264/2533] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:29 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:29.790252 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.801043 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22bf88-1b7d-4a86-9c0e-6a0a0861c7d6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.806321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.806951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.807230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.807503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.808145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.809801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.820135 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1270/2534] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:29.824209 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.835155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a6cc-1ca8-4d07-924e-ed8165f29aaa admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.840970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b69ad-09cf-4c03-b586-384486e00831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.841590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b69ad-09cf-4c03-b586-384486e00831 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.841902 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b69ad-09cf-4c03-b586-384486e00831 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.842180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b69ad-09cf-4c03-b586-384486e00831 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.842835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b69ad-09cf-4c03-b586-384486e00831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.844520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b69ad-09cf-4c03-b586-384486e00831 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:29.857210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1265/2535] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:29.861656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:29.872387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10409-766d-4f59-8bee-721754610a1e admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:29.877294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:29.878001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:29.878579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:29.878849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] PATH_INFO: `/v3/projects/ad6bbdda7d9e4a51bebde9dcaf95f93d/users/ce2d43bbdccf461d8015f5ff5893721d/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:29.897516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6bbdda7d9e4a51bebde9dcaf95f93d, user_id=ce2d43bbdccf461d8015f5ff5893721d, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:29.901004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.041342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1271/2536] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:29 2022] PUT /identity/v3/projects/ad6bbdda7d9e4a51bebde9dcaf95f93d/users/ce2d43bbdccf461d8015f5ff5893721d/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:30.056631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.057042 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.057444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.097534 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:30.108497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] MFA Rules not processed for user `ce2d43bbdccf461d8015f5ff5893721d`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:30.110595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:30.195859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:30.200481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1266/2537] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:30.212467 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:30.214390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:30.278550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb4c0c-b2f8-4d5b-8f73-2f2951d4935b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:30.282328 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b22e9b-2d8a-4d94-a60d-4a42601a8db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:30.282854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b22e9b-2d8a-4d94-a60d-4a42601a8db8 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:30.283187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b22e9b-2d8a-4d94-a60d-4a42601a8db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:30.283478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b22e9b-2d8a-4d94-a60d-4a42601a8db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:30.292533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b22e9b-2d8a-4d94-a60d-4a42601a8db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:30.294004 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b22e9b-2d8a-4d94-a60d-4a42601a8db8 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:30.305377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1272/2538] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:31.354550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:31.719293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:31.956140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:31.958133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:32.104032 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbe894-ce5a-48cc-86d3-a6bbf33aae8c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:32.108769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:32.109332 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:32.109615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:32.109934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] PATH_INFO: `/v3/users/a4c293f32c68463990ff4912a91bcd87` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:32.111112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c293f32c68463990ff4912a91bcd87)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:32.113037 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:32.385499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c293f32c68463990ff4912a91bcd87'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:32.395295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c293f32c68463990ff4912a91bcd87'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:32.621407 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c293f32c68463990ff4912a91bcd87'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:32.622081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c293f32c68463990ff4912a91bcd87'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:32.635113 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c293f32c68463990ff4912a91bcd87'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:32.641247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1267/2539] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:31 2022] DELETE /identity/v3/users/a4c293f32c68463990ff4912a91bcd87 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:32.645618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:32.646108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:32.646249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:32.675210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:32.676351 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:32.738682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1273/2540] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:32.750508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:32.751957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:32.831681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b80380-1d90-442b-964c-1e1e58fd6398 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:32.836907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:32.837533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:32.837833 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:32.838108 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:32.848551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:32.850265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:32.862089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1268/2541] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:33.445745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:33.447255 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:33.460060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8165-665f-409f-904a-e4a94703227b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:33.465713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:33.466771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.467232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.467796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] PATH_INFO: `/v3/projects/1048c52856214a36b0baa3e60ec69c84` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.469495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1048c52856214a36b0baa3e60ec69c84)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:33.472105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:33.639082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:33.679571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1048c52856214a36b0baa3e60ec69c84', 'request_id': 'req-567a0cf3-eece-486f-8afa-bc6007a5a437'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.686418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1048c52856214a36b0baa3e60ec69c84', 'request_id': 'req-567a0cf3-eece-486f-8afa-bc6007a5a437'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.692271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1048c52856214a36b0baa3e60ec69c84', 'request_id': 'req-567a0cf3-eece-486f-8afa-bc6007a5a437'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.781340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] The token cache is being invalidate because project 1048c52856214a36b0baa3e60ec69c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:33.781554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.782186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:33.783710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1274/2542] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:33 2022] DELETE /identity/v3/projects/1048c52856214a36b0baa3e60ec69c84 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:33.794343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:33.805072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87a550-2f9f-4744-8c38-d4bc03c2cf6b service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:33.844532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:33.860919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:33.861578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:33.861891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:33.862167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] PATH_INFO: `/v3/users/d96f911def6c412287a6b1842616b1c8` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:33.863315 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96f911def6c412287a6b1842616b1c8)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:33.865227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:34.156989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96f911def6c412287a6b1842616b1c8'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:34.165096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96f911def6c412287a6b1842616b1c8'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:34.244474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96f911def6c412287a6b1842616b1c8'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:34.245127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96f911def6c412287a6b1842616b1c8'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:34.258185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-793535e1-74ab-462b-9edb-5f741bdac371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96f911def6c412287a6b1842616b1c8'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:34.263971 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1269/2543] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:33 2022] DELETE /identity/v3/users/d96f911def6c412287a6b1842616b1c8 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:34.610511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:34.814350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:34.815910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:34.829454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a0cf3-eece-486f-8afa-bc6007a5a437 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:34.891887 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:34.893623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:34.894134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:34.894518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:34.894864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] PATH_INFO: `/v3/projects/c69a4801b3be4766ab28ea7853e15056` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:34.896327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69a4801b3be4766ab28ea7853e15056)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:34.898385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.131275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69a4801b3be4766ab28ea7853e15056', 'request_id': 'req-3c2341b4-872a-4b20-846f-bf864f55ee7f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:35.138432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69a4801b3be4766ab28ea7853e15056', 'request_id': 'req-3c2341b4-872a-4b20-846f-bf864f55ee7f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:35.144809 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69a4801b3be4766ab28ea7853e15056', 'request_id': 'req-3c2341b4-872a-4b20-846f-bf864f55ee7f'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:35.312937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] The token cache is being invalidate because project c69a4801b3be4766ab28ea7853e15056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:35.313169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:35.313799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-3c2341b4-872a-4b20-846f-bf864f55ee7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:35.315474 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1275/2544] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:34 2022] DELETE /identity/v3/projects/c69a4801b3be4766ab28ea7853e15056 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:35.320848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f908-b38f-4253-83a4-8447b67fe9eb None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.321282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f908-b38f-4253-83a4-8447b67fe9eb None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.321428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f908-b38f-4253-83a4-8447b67fe9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.371358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-2c63f908-b38f-4253-83a4-8447b67fe9eb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:35.372584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-2c63f908-b38f-4253-83a4-8447b67fe9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:35.434863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1270/2545] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:35.442906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1845b-3c48-4581-b9ad-7da8ff9cd0f5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.443278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1845b-3c48-4581-b9ad-7da8ff9cd0f5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.443717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1845b-3c48-4581-b9ad-7da8ff9cd0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.464422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-87b1845b-3c48-4581-b9ad-7da8ff9cd0f5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:35.465621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-87b1845b-3c48-4581-b9ad-7da8ff9cd0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:35.473737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1276/2546] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:35.478598 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f908-b38f-4253-83a4-8447b67fe9eb None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.488169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f908-b38f-4253-83a4-8447b67fe9eb None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.493249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.493881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.494180 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.494447 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.495089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:35.495389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.496977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.510356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1271/2547] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:35.514608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1845b-3c48-4581-b9ad-7da8ff9cd0f5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.524625 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1845b-3c48-4581-b9ad-7da8ff9cd0f5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.529321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.529959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.530238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.530528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.531189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.533068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.646523 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1277/2548] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:35 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:35.651441 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.661759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd967433-b38a-4391-9d97-49a1aca3bc95 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.717717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.718356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.718634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.718908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.719581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.721538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.828292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1272/2549] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:35 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:35.832594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.841015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:35.844885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1841d70-1ffa-4eb8-8673-032b4aa04cb6 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.850109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.851009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.851518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.854392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.855092 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.856945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.868134 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1278/2550] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:35.872294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.883327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e1f74-eef3-4249-a2d6-e9feaaacff9e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.888082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1ce32-2065-41e1-b06c-c1d4e67f8ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.888727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ce32-2065-41e1-b06c-c1d4e67f8ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.889000 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ce32-2065-41e1-b06c-c1d4e67f8ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.889267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ce32-2065-41e1-b06c-c1d4e67f8ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.889942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1ce32-2065-41e1-b06c-c1d4e67f8ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.891531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1ce32-2065-41e1-b06c-c1d4e67f8ca6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:35.902931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1273/2551] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:35.907102 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:35.917542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80fdc5-c053-486b-839e-b70743c68e06 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:35.922112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:35.922784 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:35.923088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:35.923357 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] PATH_INFO: `/v3/projects/79e3ab2714424ebab46ca00af7a42ac9/users/3af89012188143b6a008579cfeb57d58/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:35.939833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e3ab2714424ebab46ca00af7a42ac9, user_id=3af89012188143b6a008579cfeb57d58, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:35.942438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:36.130726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1279/2552] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:35 2022] PUT /identity/v3/projects/79e3ab2714424ebab46ca00af7a42ac9/users/3af89012188143b6a008579cfeb57d58/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:36.170900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:36.199147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987d510-f17f-4d5e-a9d9-166045400395 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.199534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987d510-f17f-4d5e-a9d9-166045400395 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.199751 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987d510-f17f-4d5e-a9d9-166045400395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.228997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f987d510-f17f-4d5e-a9d9-166045400395 None None] MFA Rules not processed for user `3af89012188143b6a008579cfeb57d58`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:36.230251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f987d510-f17f-4d5e-a9d9-166045400395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:36.303792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1274/2553] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:36.309035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:36.381365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f46dc2-eb97-4882-8bf7-7b7319c3982c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:36.386970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:36.387631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.387906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.388176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.388829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:36.390917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:36.676289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1280/2554] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:36 2022] POST /identity/v3/projects => generated 379 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:36.680617 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987d510-f17f-4d5e-a9d9-166045400395 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:36.691670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987d510-f17f-4d5e-a9d9-166045400395 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:36.747520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:36.748155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.748438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.748710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.749353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:36.751330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:36.868266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1275/2555] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:36 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:36.872394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:36.884059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3fb90-d9b7-4971-94f6-a99a8a36fbe8 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:36.888555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:36.889216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.889493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.889781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.890433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:36.892023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:36.903599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1281/2556] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:36.908397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:36.908558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:36.919476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7914e2-a635-4726-9cf0-a46106fc9098 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:36.924581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:36.925883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:36.925883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:36.926334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] PATH_INFO: `/v3/projects/ebc5552cad7d4f9fa90e1548ab2e6633/users/8302a9156e8d49d8bdabca293e038510/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:36.947043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc5552cad7d4f9fa90e1548ab2e6633, user_id=8302a9156e8d49d8bdabca293e038510, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:36.949611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:37.122104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1276/2557] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:36 2022] PUT /identity/v3/projects/ebc5552cad7d4f9fa90e1548ab2e6633/users/8302a9156e8d49d8bdabca293e038510/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:37.128955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:37.139341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18ccf6-26e9-4a56-b524-5fc23bea9b84 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:37.201461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df1dcd-7590-4ba2-b97a-b9db53961539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:37.202046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df1dcd-7590-4ba2-b97a-b9db53961539 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:37.202331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df1dcd-7590-4ba2-b97a-b9db53961539 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:37.202617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df1dcd-7590-4ba2-b97a-b9db53961539 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:37.203329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df1dcd-7590-4ba2-b97a-b9db53961539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:37.204957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df1dcd-7590-4ba2-b97a-b9db53961539 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:37.221471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1282/2558] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:37.222832 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:37.234935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94655ff-8eea-4cd9-b47f-9f859c77c76e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:37.242579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:37.243472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:37.243824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:37.244168 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] PATH_INFO: `/v3/projects/ebc5552cad7d4f9fa90e1548ab2e6633/users/8302a9156e8d49d8bdabca293e038510/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:37.246484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc5552cad7d4f9fa90e1548ab2e6633, user_id=8302a9156e8d49d8bdabca293e038510, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:37.249698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:37.263487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1277/2559] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:37 2022] PUT /identity/v3/projects/ebc5552cad7d4f9fa90e1548ab2e6633/users/8302a9156e8d49d8bdabca293e038510/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:37.275226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:37.275509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:37.275780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:37.309829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] MFA Rules not processed for user `8302a9156e8d49d8bdabca293e038510`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:37.311068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:37.398612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1283/2560] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:37.410238 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:37.411968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:37.473695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01eb09-6273-4004-bf5a-e77616bc2289 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:37.478263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:37.478886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:37.479169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:37.491284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:37.491284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:37.491284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:37.499754 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1278/2561] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:37.846224 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:37.847670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:37.918948 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818c4fd-5966-4612-9e5d-adb4fb85b7ce None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:37.924739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:37.925395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:37.925812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:37.925984 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] PATH_INFO: `/v3/users/13f2f26e9daf416ba46371b5948ab743` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:37.927235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f2f26e9daf416ba46371b5948ab743)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:37.929728 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:38.364988 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f2f26e9daf416ba46371b5948ab743'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.373103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:38.378903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f2f26e9daf416ba46371b5948ab743'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.379822 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f2f26e9daf416ba46371b5948ab743'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.380828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f2f26e9daf416ba46371b5948ab743'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.384818 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f399-2f6a-48f3-9a4d-e1fcf4485d39 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:38.394155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:38.394155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:38.394155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:38.394155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] PATH_INFO: `/v3/users/3af89012188143b6a008579cfeb57d58` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:38.394607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af89012188143b6a008579cfeb57d58)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:38.394607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:38.501930 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-70478355-ac49-466a-8ffd-13dc7937713a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f2f26e9daf416ba46371b5948ab743'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.525856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1284/2562] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:37 2022] DELETE /identity/v3/users/13f2f26e9daf416ba46371b5948ab743 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:38.535432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:38.538751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:38.538751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:38.559871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:38.561143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:38.622480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1285/2563] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:38.633199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:38.634673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:38.694573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa03c3-dae7-4410-93fb-e418f20f623b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:38.699615 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0165d-7488-4c16-9fd9-86a8435a169d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:38.700231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0165d-7488-4c16-9fd9-86a8435a169d service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:38.700541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0165d-7488-4c16-9fd9-86a8435a169d service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:38.700823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0165d-7488-4c16-9fd9-86a8435a169d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:38.707258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af89012188143b6a008579cfeb57d58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.714203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0165d-7488-4c16-9fd9-86a8435a169d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:38.714203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0165d-7488-4c16-9fd9-86a8435a169d service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:38.715172 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af89012188143b6a008579cfeb57d58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.723289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1286/2564] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:38.848568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af89012188143b6a008579cfeb57d58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.848568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af89012188143b6a008579cfeb57d58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.867317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af89012188143b6a008579cfeb57d58'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:38.873664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1279/2565] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:38 2022] DELETE /identity/v3/users/3af89012188143b6a008579cfeb57d58 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:38.879040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:38.879040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:38.879271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:38.904158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:38.904158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:38.965903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1287/2566] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:38.975917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:38.977341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:39.040325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d18375-bbc2-4826-82e5-c109bde765b0 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:39.044352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:39.045010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:39.045293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:39.045568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:39.056661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:39.058181 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:39.071436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1280/2567] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:39.262178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:39.529399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:39.530985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:39.544754 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4703f-811b-4624-9df1-7a0a680247f3 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:39.564008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:39.564764 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:39.564970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:39.565244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] PATH_INFO: `/v3/projects/a2a8b05bca8d4a30ad3286d5a10ad923` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:39.569160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a8b05bca8d4a30ad3286d5a10ad923)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:39.574838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:39.825993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a8b05bca8d4a30ad3286d5a10ad923', 'request_id': 'req-73690fef-b7ff-46b3-a562-c4d5c47e3042'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:39.833564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a8b05bca8d4a30ad3286d5a10ad923', 'request_id': 'req-73690fef-b7ff-46b3-a562-c4d5c47e3042'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:39.846202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a8b05bca8d4a30ad3286d5a10ad923', 'request_id': 'req-73690fef-b7ff-46b3-a562-c4d5c47e3042'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:39.863563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:39.866045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:39.904430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb281cfb-7ef8-4475-b5c2-4b7937d9596b service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:39.948179 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:39.971188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:39.971915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:39.972194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:39.972662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] PATH_INFO: `/v3/projects/79e3ab2714424ebab46ca00af7a42ac9` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:39.973995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e3ab2714424ebab46ca00af7a42ac9)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:39.975984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:39.989557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] The token cache is being invalidate because project a2a8b05bca8d4a30ad3286d5a10ad923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:39.993941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:39.993941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-73690fef-b7ff-46b3-a562-c4d5c47e3042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:39.993941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1288/2568] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:39 2022] DELETE /identity/v3/projects/a2a8b05bca8d4a30ad3286d5a10ad923 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:40.001516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72918aee-ee06-4f61-914c-ac1132601fe2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.002013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72918aee-ee06-4f61-914c-ac1132601fe2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.002426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72918aee-ee06-4f61-914c-ac1132601fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.037324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-72918aee-ee06-4f61-914c-ac1132601fe2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:40.039385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-72918aee-ee06-4f61-914c-ac1132601fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:40.048424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1289/2569] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:40.056401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40264b-1e3f-4679-a7c3-53e8791a9dbd None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.056685 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40264b-1e3f-4679-a7c3-53e8791a9dbd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.064717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40264b-1e3f-4679-a7c3-53e8791a9dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.078587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-1d40264b-1e3f-4679-a7c3-53e8791a9dbd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:40.079724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1d40264b-1e3f-4679-a7c3-53e8791a9dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:40.088507 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1290/2570] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:40.092490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40264b-1e3f-4679-a7c3-53e8791a9dbd None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.101658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40264b-1e3f-4679-a7c3-53e8791a9dbd None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.106343 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.110602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.110602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.110602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.110602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:40.110602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:40.110602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:40.122311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1291/2571] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:08:40.127109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.138579 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ec5bf-1f0b-4f27-a2f6-983daa7ea344 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.143502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.144245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.144546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.144821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.145446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:40.147438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:40.248347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e3ab2714424ebab46ca00af7a42ac9', 'request_id': 'req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:40.255240 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e3ab2714424ebab46ca00af7a42ac9', 'request_id': 'req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:40.490536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1292/2572] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:40 2022] POST /identity/v3/projects => generated 365 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:40.492661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.560077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e3ab2714424ebab46ca00af7a42ac9', 'request_id': 'req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:40.562681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066dede-b43f-44e3-a8dc-5463d7366805 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.569173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.570014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.570650 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.571045 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.571792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:40.573880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:40.580768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] The token cache is being invalidate because project 79e3ab2714424ebab46ca00af7a42ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:40.581388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:40.582329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:40.587755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1281/2573] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:39 2022] DELETE /identity/v3/projects/79e3ab2714424ebab46ca00af7a42ac9 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:40.588988 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.590830 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:40.681208 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d7bf6-fb4e-49e6-a3b1-d03da6d99058 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.688218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.688218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.688218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.688218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] PATH_INFO: `/v3/users/8302a9156e8d49d8bdabca293e038510` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.690417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8302a9156e8d49d8bdabca293e038510)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:40.692904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:40.848530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1293/2574] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:40 2022] POST /identity/v3/users => generated 283 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:40.849673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.851978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:40.872044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591dea5-6690-442f-9a45-6ae17bc008e1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.889702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.895725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.896015 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.896302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.899031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:40.903032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:40.916125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1294/2575] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:40.921063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.936340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f8c1d-a10c-4141-af26-81fa81ec7343 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.944072 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.945968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.946628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.946628 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:40.948326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:40.949660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:40.970364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1295/2576] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:40.973162 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:40.987648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a028083-86f0-4bc7-a309-f6996a20dde1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:40.993193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6a963-a4b8-4f2c-a8fd-96560b5f6e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:40.993951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6a963-a4b8-4f2c-a8fd-96560b5f6e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:40.994195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6a963-a4b8-4f2c-a8fd-96560b5f6e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:40.994623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6a963-a4b8-4f2c-a8fd-96560b5f6e67 admin admin] PATH_INFO: `/v3/projects/214bbb12729e47c5876c4ed7dd435672/users/9135eae5264541b49d5bb6507bc81f3d/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:41.031313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6a963-a4b8-4f2c-a8fd-96560b5f6e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=214bbb12729e47c5876c4ed7dd435672, user_id=9135eae5264541b49d5bb6507bc81f3d, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:41.034008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6a963-a4b8-4f2c-a8fd-96560b5f6e67 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:41.258509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1296/2577] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:40 2022] PUT /identity/v3/projects/214bbb12729e47c5876c4ed7dd435672/users/9135eae5264541b49d5bb6507bc81f3d/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:41.268896 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:41.269428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:41.269882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:41.300817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] MFA Rules not processed for user `9135eae5264541b49d5bb6507bc81f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:41.302110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:41.346552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8302a9156e8d49d8bdabca293e038510'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:41.356018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8302a9156e8d49d8bdabca293e038510'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:41.383651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1297/2578] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:41.394853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:41.396261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:41.455502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb32f0d-f685-4105-bb6c-056c6af61b9b None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:41.459408 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:41.460166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:41.461013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:41.461013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:41.470141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:41.471712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:41.482591 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1298/2579] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:41.604494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8302a9156e8d49d8bdabca293e038510'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:41.604494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8302a9156e8d49d8bdabca293e038510'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:41.620564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8302a9156e8d49d8bdabca293e038510'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:41.627347 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1282/2580] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:40 2022] DELETE /identity/v3/users/8302a9156e8d49d8bdabca293e038510 => generated 0 bytes in 1039 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:42.273640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:42.353347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160bccd-193c-40f3-9b40-c38a6eddd955 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:42.418575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:42.424989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:42.427720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:42.428369 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] PATH_INFO: `/v3/projects/ebc5552cad7d4f9fa90e1548ab2e6633` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:42.431124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc5552cad7d4f9fa90e1548ab2e6633)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:42.436418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:42.600049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:42.601697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:42.616111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311d724-44ea-4e72-bfb4-dc0772bbc7fe admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:42.622169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:42.622805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:42.623090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:42.623365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] PATH_INFO: `/v3/users/ce2d43bbdccf461d8015f5ff5893721d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:42.624510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2d43bbdccf461d8015f5ff5893721d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:42.626487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:42.673898 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc5552cad7d4f9fa90e1548ab2e6633', 'request_id': 'req-002fd602-ac78-4df7-9402-d981e9bacdfa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:42.681140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc5552cad7d4f9fa90e1548ab2e6633', 'request_id': 'req-002fd602-ac78-4df7-9402-d981e9bacdfa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:42.687171 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc5552cad7d4f9fa90e1548ab2e6633', 'request_id': 'req-002fd602-ac78-4df7-9402-d981e9bacdfa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:42.914403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] The token cache is being invalidate because project ebc5552cad7d4f9fa90e1548ab2e6633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:42.914718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:42.915864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-002fd602-ac78-4df7-9402-d981e9bacdfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:42.917758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1299/2581] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:42 2022] DELETE /identity/v3/projects/ebc5552cad7d4f9fa90e1548ab2e6633 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:42.922613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf457-7f68-4f52-b1ef-26d4662fbcc2 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:42.923028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf457-7f68-4f52-b1ef-26d4662fbcc2 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:42.923364 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf457-7f68-4f52-b1ef-26d4662fbcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:42.949211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c3ebf457-7f68-4f52-b1ef-26d4662fbcc2 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:42.950513 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c3ebf457-7f68-4f52-b1ef-26d4662fbcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:43.019793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1300/2582] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:43.021429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03347fc-fdfb-4087-96b3-d8bfc072fe76 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.021694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03347fc-fdfb-4087-96b3-d8bfc072fe76 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.021990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03347fc-fdfb-4087-96b3-d8bfc072fe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.046315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c03347fc-fdfb-4087-96b3-d8bfc072fe76 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:43.047510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c03347fc-fdfb-4087-96b3-d8bfc072fe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:43.055486 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1301/2583] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:43.057996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2d43bbdccf461d8015f5ff5893721d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:43.059656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03347fc-fdfb-4087-96b3-d8bfc072fe76 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:43.065303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2d43bbdccf461d8015f5ff5893721d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:43.068459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03347fc-fdfb-4087-96b3-d8bfc072fe76 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:43.074014 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:43.074610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.074916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.075208 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.075855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:43.076183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:43.077833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:43.090103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1302/2584] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:08:43.104481 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:43.114112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213d1e-d3fe-419e-b0e3-0e9b58f863e7 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:43.119028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:43.119617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.119895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.120167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.120796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:43.122761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:43.198966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2d43bbdccf461d8015f5ff5893721d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:43.199695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2d43bbdccf461d8015f5ff5893721d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:43.213277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-41dcadd5-202a-4487-bbf4-b09d35f4213f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2d43bbdccf461d8015f5ff5893721d'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:43.219252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1283/2585] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:42 2022] DELETE /identity/v3/users/ce2d43bbdccf461d8015f5ff5893721d => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:43.223776 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.224127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.224425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.246161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:43.247433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:43.318442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1284/2586] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:43.325123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:43.326626 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:43.343874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1303/2587] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:43 2022] POST /identity/v3/projects => generated 363 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:43.348469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:43.350504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:43.394030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be210f05-e88d-45ce-b451-58c3fa46a8cd None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:43.429595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca890f-533e-4631-b1dc-4e5ed415cbbb admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:43.438627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:43.439370 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.439792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.440070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.440714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:43.443121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:43.450322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:43.450965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.451275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.451545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.463204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:43.465607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:43.477133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1285/2588] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:43.662959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1304/2589] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:43 2022] POST /identity/v3/users => generated 282 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:43.666865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:43.676733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ebc9-5194-43be-9e6d-cb30117e7725 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:43.681378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:43.682014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.682296 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.682630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.683510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:43.685259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:43.696628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1286/2590] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:43.700820 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:43.711793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2445-2535-45c9-9e36-fbe2ecaeede1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:43.720957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b967d1c-a46e-46ed-a38a-8819d623f0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:43.720957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b967d1c-a46e-46ed-a38a-8819d623f0fc admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.720957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b967d1c-a46e-46ed-a38a-8819d623f0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.720957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b967d1c-a46e-46ed-a38a-8819d623f0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.721387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b967d1c-a46e-46ed-a38a-8819d623f0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:43.721387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b967d1c-a46e-46ed-a38a-8819d623f0fc admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:43.730521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1305/2591] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:43.734706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:43.745299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f8617-6880-4331-8e13-03bc9483ad6e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:43.749692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:43.750402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:43.750682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:43.750955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] PATH_INFO: `/v3/projects/5191ebeda1ab4de3901f00fee0a7be3e/users/7d6131abf44e46a5b7fe076cf4173757/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:43.767393 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5191ebeda1ab4de3901f00fee0a7be3e, user_id=7d6131abf44e46a5b7fe076cf4173757, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:43.770062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:44.120904 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1287/2592] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:43 2022] PUT /identity/v3/projects/5191ebeda1ab4de3901f00fee0a7be3e/users/7d6131abf44e46a5b7fe076cf4173757/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:44.133566 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:44.133983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:44.134272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:44.166499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] MFA Rules not processed for user `7d6131abf44e46a5b7fe076cf4173757`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:44.167911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:44.180738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:44.183194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:44.265506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1306/2593] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:44.279276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb26b7c-04b5-4993-be6c-307b510cec51 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:44.283851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:44.284565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:44.284845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:44.285121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] PATH_INFO: `/v3/projects/ad6bbdda7d9e4a51bebde9dcaf95f93d` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:44.286400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6bbdda7d9e4a51bebde9dcaf95f93d)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:44.289338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:44.290395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:44.290915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:44.362660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10840d6a-c01f-491a-b9f1-80fb977333fd None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:44.367247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13098549-c5e1-4cf1-9019-fd00bb41c4ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:44.374090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13098549-c5e1-4cf1-9019-fd00bb41c4ca service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:44.374090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13098549-c5e1-4cf1-9019-fd00bb41c4ca service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:44.374090 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13098549-c5e1-4cf1-9019-fd00bb41c4ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:44.387054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13098549-c5e1-4cf1-9019-fd00bb41c4ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:44.388664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13098549-c5e1-4cf1-9019-fd00bb41c4ca service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:44.400386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1307/2594] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:44.515759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6bbdda7d9e4a51bebde9dcaf95f93d', 'request_id': 'req-89208168-dab2-487d-b354-91587c4460c9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:44.522220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6bbdda7d9e4a51bebde9dcaf95f93d', 'request_id': 'req-89208168-dab2-487d-b354-91587c4460c9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:44.784807 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6bbdda7d9e4a51bebde9dcaf95f93d', 'request_id': 'req-89208168-dab2-487d-b354-91587c4460c9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:44.800245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] The token cache is being invalidate because project ad6bbdda7d9e4a51bebde9dcaf95f93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:44.800547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:44.801217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-89208168-dab2-487d-b354-91587c4460c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:44.806646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1288/2595] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:44 2022] DELETE /identity/v3/projects/ad6bbdda7d9e4a51bebde9dcaf95f93d => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:44.808081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb723c-3e28-45ed-910a-d691f5d2eb38 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:44.808380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb723c-3e28-45ed-910a-d691f5d2eb38 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:44.808751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb723c-3e28-45ed-910a-d691f5d2eb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:44.829249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-0ddb723c-3e28-45ed-910a-d691f5d2eb38 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:44.831156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-0ddb723c-3e28-45ed-910a-d691f5d2eb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:44.895365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1308/2596] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:44.903011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969eccd-a736-4bf5-8b04-d22161fb5243 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:44.903303 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969eccd-a736-4bf5-8b04-d22161fb5243 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:44.903586 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969eccd-a736-4bf5-8b04-d22161fb5243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:44.923167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b969eccd-a736-4bf5-8b04-d22161fb5243 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:44.924394 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b969eccd-a736-4bf5-8b04-d22161fb5243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:44.932824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1289/2597] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:44.937220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb723c-3e28-45ed-910a-d691f5d2eb38 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:44.948067 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb723c-3e28-45ed-910a-d691f5d2eb38 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:44.952805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:44.953437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:44.953708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:44.954002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:44.954667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:44.954954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:44.956586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:44.968261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1309/2598] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:44.972982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969eccd-a736-4bf5-8b04-d22161fb5243 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:44.983097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969eccd-a736-4bf5-8b04-d22161fb5243 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:44.988092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:44.988714 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:44.989029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:44.989270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:44.989951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:44.992380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:45.292397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1290/2599] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:44 2022] POST /identity/v3/projects => generated 375 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:45.296587 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:45.320528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc6f1-e139-46ee-aba4-3b454ed0ab8d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:45.383613 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:45.384249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:45.384478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:45.384808 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:45.385361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:45.387327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:45.618990 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1310/2600] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:45 2022] POST /identity/v3/users => generated 288 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:45.631478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:45.635437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e0a4-56ff-4cef-b090-3276c860fb18 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:45.640460 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:45.641259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:45.641537 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:45.641836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:45.642521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:45.644179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:45.655323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1291/2601] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:45.659724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:45.671183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55825-b820-41e1-940e-aa844b7afe39 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:45.676511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da3a02-573e-4b11-b15d-ec6ef26538d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:45.677204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da3a02-573e-4b11-b15d-ec6ef26538d8 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:45.677516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da3a02-573e-4b11-b15d-ec6ef26538d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:45.677860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da3a02-573e-4b11-b15d-ec6ef26538d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:45.678557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da3a02-573e-4b11-b15d-ec6ef26538d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:45.680905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da3a02-573e-4b11-b15d-ec6ef26538d8 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:45.693295 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1311/2602] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:45.698702 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:45.709396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c47bc-bd34-4946-a1eb-472cd8eee9df admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:45.714364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:45.715017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:45.715295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:45.715582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] PATH_INFO: `/v3/projects/c8a99dc23569430092186b6aa784e2f4/users/203400515a2d4588913d058294b0279d/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:45.732624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a99dc23569430092186b6aa784e2f4, user_id=203400515a2d4588913d058294b0279d, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:45.735469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:45.924424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:46.001645 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1292/2603] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:45 2022] PUT /identity/v3/projects/c8a99dc23569430092186b6aa784e2f4/users/203400515a2d4588913d058294b0279d/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:46.015218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcd562-4167-4405-8697-6a24b9f2ed48 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.015877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcd562-4167-4405-8697-6a24b9f2ed48 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.016324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcd562-4167-4405-8697-6a24b9f2ed48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.046998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e9dcd562-4167-4405-8697-6a24b9f2ed48 None None] MFA Rules not processed for user `203400515a2d4588913d058294b0279d`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:46.048478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e9dcd562-4167-4405-8697-6a24b9f2ed48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:46.117519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1312/2604] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:46.122099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:46.185789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001deff-41ec-49d7-a597-e983f58d277f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:46.190414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:46.191022 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.191298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.191569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.192198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:46.194103 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:46.599459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1293/2605] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:46 2022] POST /identity/v3/projects => generated 375 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:46.603640 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcd562-4167-4405-8697-6a24b9f2ed48 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:46.614967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcd562-4167-4405-8697-6a24b9f2ed48 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:46.681273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:46.681941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.682285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.682604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.683247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:46.685345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:46.967665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1313/2606] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:46 2022] POST /identity/v3/users => generated 294 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:46.971192 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:46.982737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20513352-3b41-49fa-a1a5-7f5f05f238c0 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:46.987915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:46.988557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:46.992134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:46.992134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:46.992134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:46.992134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:47.003817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1294/2607] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:47.008633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:47.019341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34060b1-f8e8-41f3-91ff-cf3af90e8bb1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:47.025202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:47.025923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:47.026508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:47.026800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] PATH_INFO: `/v3/projects/365743877e224104b0b88e55a08bb59c/users/3edd4d3c87fb4ac19f2bf8009a1d0419/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:47.043472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365743877e224104b0b88e55a08bb59c, user_id=3edd4d3c87fb4ac19f2bf8009a1d0419, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:47.050280 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:47.378761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1314/2608] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:47 2022] PUT /identity/v3/projects/365743877e224104b0b88e55a08bb59c/users/3edd4d3c87fb4ac19f2bf8009a1d0419/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:47.387572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:47.401027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905396d-319c-484e-97a2-f9f04e000435 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:47.456244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:47.464416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a60fad-4574-4feb-a218-2d02883a843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:47.465036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a60fad-4574-4feb-a218-2d02883a843d admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:47.465317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a60fad-4574-4feb-a218-2d02883a843d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:47.475310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a60fad-4574-4feb-a218-2d02883a843d admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:47.475310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a60fad-4574-4feb-a218-2d02883a843d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:47.475310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a60fad-4574-4feb-a218-2d02883a843d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:47.478346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1295/2609] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:47.482659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:47.494418 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dcb20-e378-47db-9a35-2c517456ca43 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:47.499174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:47.499841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:47.500118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:47.500388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] PATH_INFO: `/v3/projects/365743877e224104b0b88e55a08bb59c/users/3edd4d3c87fb4ac19f2bf8009a1d0419/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:47.502333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=365743877e224104b0b88e55a08bb59c, user_id=3edd4d3c87fb4ac19f2bf8009a1d0419, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:47.505245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:47.516538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1315/2610] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:47 2022] PUT /identity/v3/projects/365743877e224104b0b88e55a08bb59c/users/3edd4d3c87fb4ac19f2bf8009a1d0419/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:47.542418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:47.542724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:47.542996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:47.571671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] MFA Rules not processed for user `3edd4d3c87fb4ac19f2bf8009a1d0419`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:47.572896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:47.652486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1296/2611] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:47.663464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:47.665502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:47.728935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5aa0ab-73e8-4980-95a3-0838a4653edf admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:47.732656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:47.733262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:47.733541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:47.733919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:47.794396 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:47.795881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:47.805693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1316/2612] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:48.132881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:48.135289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:48.216677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61a5d6-3124-4981-9bda-08ba2b363ef0 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:48.221199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d979c1f-b259-4e65-86cc-8f32cbedea58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:48.222186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d979c1f-b259-4e65-86cc-8f32cbedea58 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:48.222506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d979c1f-b259-4e65-86cc-8f32cbedea58 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:48.222851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d979c1f-b259-4e65-86cc-8f32cbedea58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:48.225124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1d979c1f-b259-4e65-86cc-8f32cbedea58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:48.325683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d979c1f-b259-4e65-86cc-8f32cbedea58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:48.327227 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d979c1f-b259-4e65-86cc-8f32cbedea58 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:48.338064 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1297/2613] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:52.944185 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:52.957042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22ff07-49de-4fa5-8d4f-f788ce25edf6 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:52.962444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:52.963321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:52.963792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:52.964278 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] PATH_INFO: `/v3/users/203400515a2d4588913d058294b0279d` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:52.965639 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=203400515a2d4588913d058294b0279d)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:52.967637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:53.376802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:53.447168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '203400515a2d4588913d058294b0279d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:53.457050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '203400515a2d4588913d058294b0279d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:53.457437 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '203400515a2d4588913d058294b0279d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:53.458159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '203400515a2d4588913d058294b0279d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:53.673068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '203400515a2d4588913d058294b0279d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:53.692539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1317/2614] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:52 2022] DELETE /identity/v3/users/203400515a2d4588913d058294b0279d => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:53.698854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806f68-32b5-4917-940e-435aac702e5b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:53.699137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806f68-32b5-4917-940e-435aac702e5b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:53.699404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806f68-32b5-4917-940e-435aac702e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:53.718481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-99806f68-32b5-4917-940e-435aac702e5b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:53.719698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-99806f68-32b5-4917-940e-435aac702e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:53.780439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1298/2615] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:53.790247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:53.791722 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:53.916325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875153fb-87e2-4224-854f-d3da8e528df0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:53.921485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:53.921925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:53.922406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:53.922922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:53.939075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:53.940830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:53.951740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1318/2616] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:54.349435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99806f68-32b5-4917-940e-435aac702e5b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:54.351435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-99806f68-32b5-4917-940e-435aac702e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:54.365591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99806f68-32b5-4917-940e-435aac702e5b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:54.372780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:54.373525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:54.373905 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:54.374293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] PATH_INFO: `/v3/users/b77f38582b8842d7a303e8431299191a` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:54.375891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77f38582b8842d7a303e8431299191a)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:54.379234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:54.839425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:54.847233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:54.848989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:54.857240 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77f38582b8842d7a303e8431299191a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:54.870875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9de95-bf25-494b-9ce7-5256ba030120 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:54.871933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77f38582b8842d7a303e8431299191a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:54.876393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:54.877046 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:54.877432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:54.877574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] PATH_INFO: `/v3/projects/c8a99dc23569430092186b6aa784e2f4` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:54.879041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a99dc23569430092186b6aa784e2f4)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:54.881425 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:55.126431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77f38582b8842d7a303e8431299191a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.127607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77f38582b8842d7a303e8431299191a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.146289 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a99dc23569430092186b6aa784e2f4', 'request_id': 'req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.146844 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77f38582b8842d7a303e8431299191a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.154509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1299/2617] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:54 2022] DELETE /identity/v3/users/b77f38582b8842d7a303e8431299191a => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:55.155144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a99dc23569430092186b6aa784e2f4', 'request_id': 'req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.160983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a99dc23569430092186b6aa784e2f4', 'request_id': 'req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.461461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] The token cache is being invalidate because project c8a99dc23569430092186b6aa784e2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:55.461752 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.462426 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:55.463934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1319/2618] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:54 2022] DELETE /identity/v3/projects/c8a99dc23569430092186b6aa784e2f4 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:55.468495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:55.469854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:55.537259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c024227-e04d-48d7-a168-bc91bfbfdde4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:55.545639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:55.545639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:55.545639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:55.545639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] PATH_INFO: `/v3/users/3edd4d3c87fb4ac19f2bf8009a1d0419` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:55.546424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edd4d3c87fb4ac19f2bf8009a1d0419)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:55.548051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:55.617411 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:55.619106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:55.687960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1abf3-5de0-4c28-a16d-4cebf8a393e5 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:55.692915 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:55.693610 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:55.693945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:55.694876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] PATH_INFO: `/v3/projects/7e322f2f7da74039aff258b70af12095` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:55.696023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e322f2f7da74039aff258b70af12095)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:55.698012 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:56.012459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edd4d3c87fb4ac19f2bf8009a1d0419'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.019688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edd4d3c87fb4ac19f2bf8009a1d0419'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.021363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e322f2f7da74039aff258b70af12095', 'request_id': 'req-14491be8-aebd-4e43-86d2-70d3eb1b5e52'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.028006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e322f2f7da74039aff258b70af12095', 'request_id': 'req-14491be8-aebd-4e43-86d2-70d3eb1b5e52'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.033817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e322f2f7da74039aff258b70af12095', 'request_id': 'req-14491be8-aebd-4e43-86d2-70d3eb1b5e52'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.136090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edd4d3c87fb4ac19f2bf8009a1d0419'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.136816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edd4d3c87fb4ac19f2bf8009a1d0419'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.151044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0aadf992-dd1f-4dd0-bcd0-fa9387f1e212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edd4d3c87fb4ac19f2bf8009a1d0419'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.160731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1300/2619] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:55 2022] DELETE /identity/v3/users/3edd4d3c87fb4ac19f2bf8009a1d0419 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:56.341274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] The token cache is being invalidate because project 7e322f2f7da74039aff258b70af12095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:56.341891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.342479 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-14491be8-aebd-4e43-86d2-70d3eb1b5e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.344113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1320/2620] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:55 2022] DELETE /identity/v3/projects/7e322f2f7da74039aff258b70af12095 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:56.349204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922775-e4ed-4152-8024-12c16fbef3a1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.349528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922775-e4ed-4152-8024-12c16fbef3a1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.356012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922775-e4ed-4152-8024-12c16fbef3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.375697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b1922775-e4ed-4152-8024-12c16fbef3a1 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:56.376919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b1922775-e4ed-4152-8024-12c16fbef3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:56.453254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1301/2621] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:56.462464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704772ce-b5e3-4fcf-b3d6-0af6e781eb93 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.462758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704772ce-b5e3-4fcf-b3d6-0af6e781eb93 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.468244 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704772ce-b5e3-4fcf-b3d6-0af6e781eb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.485197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-704772ce-b5e3-4fcf-b3d6-0af6e781eb93 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:56.487491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-704772ce-b5e3-4fcf-b3d6-0af6e781eb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:56.497531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1321/2622] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:56.514013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1922775-e4ed-4152-8024-12c16fbef3a1 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:56.524158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1922775-e4ed-4152-8024-12c16fbef3a1 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:56.529224 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:56.529873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.530154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.530603 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.531252 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:56.531562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:56.533160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:56.546910 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1302/2623] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:56.552008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704772ce-b5e3-4fcf-b3d6-0af6e781eb93 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:56.563329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704772ce-b5e3-4fcf-b3d6-0af6e781eb93 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:56.568920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:56.579293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.579293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.579293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.579293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:56.579293 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:56.771994 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:56.780722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:56.792943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a91ad-8527-4019-ad05-d2a932cdaea0 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:56.798106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:56.798762 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.799043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.799367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] PATH_INFO: `/v3/projects/365743877e224104b0b88e55a08bb59c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.800463 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=365743877e224104b0b88e55a08bb59c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:56.802806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:56.817583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1322/2624] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:56 2022] POST /identity/v3/projects => generated 381 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:56.818427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:56.888553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485b887-0d8c-48ac-b294-f468cb9286e7 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:56.893243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:56.893880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:56.894213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:56.894533 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:56.895164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:56.897160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:56.966391 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365743877e224104b0b88e55a08bb59c', 'request_id': 'req-66883797-8d6b-4c81-a56c-f31148866f7b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:56.972039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365743877e224104b0b88e55a08bb59c', 'request_id': 'req-66883797-8d6b-4c81-a56c-f31148866f7b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:57.005227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1323/2625] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:56 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:57.018796 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.029424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e917ea-1a2e-4d56-b617-420858c931a9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.089146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.089802 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.090211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.090720 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.091386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.093049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.103631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1324/2626] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:57.107548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365743877e224104b0b88e55a08bb59c', 'request_id': 'req-66883797-8d6b-4c81-a56c-f31148866f7b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:57.108210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.118287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f197b-0ad7-42af-b501-80c0af404b12 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.120110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] The token cache is being invalidate because project 365743877e224104b0b88e55a08bb59c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:08:57.120364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:57.121020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-66883797-8d6b-4c81-a56c-f31148866f7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:57.122720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1303/2627] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:56 2022] DELETE /identity/v3/projects/365743877e224104b0b88e55a08bb59c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:57.123718 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.124344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.124634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.124912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.125596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.127518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.132895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb9129-31d6-45e9-adcd-9a2d5c64fcad None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.132895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb9129-31d6-45e9-adcd-9a2d5c64fcad None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.132895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb9129-31d6-45e9-adcd-9a2d5c64fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.141607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1325/2628] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:57.146319 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.149565 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:57.153392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0aeb9129-31d6-45e9-adcd-9a2d5c64fcad None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:57.154658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0aeb9129-31d6-45e9-adcd-9a2d5c64fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:57.163486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1304/2629] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:57.169866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cdc37-e748-47e3-8bd2-195f27dc6ab0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.171438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9928-ff75-4d64-ad08-b81719c6cb60 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.171796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9928-ff75-4d64-ad08-b81719c6cb60 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.172080 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9928-ff75-4d64-ad08-b81719c6cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.175880 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f025af-2b03-406d-873a-6feafcb9ff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.176790 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f025af-2b03-406d-873a-6feafcb9ff3d admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.177211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f025af-2b03-406d-873a-6feafcb9ff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.177624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f025af-2b03-406d-873a-6feafcb9ff3d admin admin] PATH_INFO: `/v3/projects/7a48453b4f0e48e5be4d83ced4691e24/users/3e8d6662fc0b4f35bdc7ea178d7eb0ba/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.193737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-561c9928-ff75-4d64-ad08-b81719c6cb60 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:57.203122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-561c9928-ff75-4d64-ad08-b81719c6cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:57.203261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f025af-2b03-406d-873a-6feafcb9ff3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a48453b4f0e48e5be4d83ced4691e24, user_id=3e8d6662fc0b4f35bdc7ea178d7eb0ba, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.203261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f025af-2b03-406d-873a-6feafcb9ff3d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.204816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1305/2630] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:57.209085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9928-ff75-4d64-ad08-b81719c6cb60 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.219154 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9928-ff75-4d64-ad08-b81719c6cb60 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.223982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.224576 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.224852 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.225123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.225883 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:08:57.226187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.227782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.239201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1306/2631] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:08:57.243533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.254098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1039bc6-6753-45bd-92e7-19f241a55535 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.259247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.259848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.260204 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.260476 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.261120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.263014 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.307546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1326/2632] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:57 2022] PUT /identity/v3/projects/7a48453b4f0e48e5be4d83ced4691e24/users/3e8d6662fc0b4f35bdc7ea178d7eb0ba/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:57.318997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d055d35-8e77-4c35-a53d-590904509c17 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.319775 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d055d35-8e77-4c35-a53d-590904509c17 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.320042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d055d35-8e77-4c35-a53d-590904509c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.348637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4d055d35-8e77-4c35-a53d-590904509c17 None None] MFA Rules not processed for user `3e8d6662fc0b4f35bdc7ea178d7eb0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:57.349934 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4d055d35-8e77-4c35-a53d-590904509c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:57.416106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1327/2633] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:08:57.424937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d055d35-8e77-4c35-a53d-590904509c17 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.436871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1307/2634] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:57.442100 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.490916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d055d35-8e77-4c35-a53d-590904509c17 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.508386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29830ef2-6a75-4f9c-a337-10318a9d64f6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.513420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.514221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.514831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.514831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.515384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.517434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.555595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.556287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.556580 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.556857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.557496 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.561495 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.638961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1308/2635] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:57.643403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.653572 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd06dd-3954-4047-ac3d-010f78cd75d2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.658388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.658977 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.659253 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.659520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.660210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.661800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.676680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1309/2636] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:57.677592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.682801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1328/2637] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:08:57.686658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.689166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc6f71-36e7-448b-a554-501e117fc8b0 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.747761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.748531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.748814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.749087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.749731 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.750978 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb5678-c6d9-426b-b6b8-3721627475bd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.751364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.756201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.756989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.757457 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.757897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.758876 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.760940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.763287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1310/2638] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:57.767481 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.778104 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b2b82-f3d7-4bdf-8c85-57df1df7432a admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.782997 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930a9d4-04de-4278-b0eb-983140a4107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.783651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930a9d4-04de-4278-b0eb-983140a4107e admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.783933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930a9d4-04de-4278-b0eb-983140a4107e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.784215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930a9d4-04de-4278-b0eb-983140a4107e admin admin] PATH_INFO: `/v3/projects/3ccbbaaf9a874676850c7ba33b832bd0/users/bf77fcc833374b6fa165457bcf647648/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.800671 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930a9d4-04de-4278-b0eb-983140a4107e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ccbbaaf9a874676850c7ba33b832bd0, user_id=bf77fcc833374b6fa165457bcf647648, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.803580 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930a9d4-04de-4278-b0eb-983140a4107e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.924402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1329/2639] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:08:57.928262 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.938415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22858-26fd-4513-8415-c42dcc1e104b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.943125 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.943716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.943995 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.944271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.944926 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:57.946597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:57.957873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1330/2640] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:08:57.962077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:57.972201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6672e-cc3e-4ccb-b4af-be12ad5d1025 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:57.977480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:57.978176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:57.978458 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:57.978734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] PATH_INFO: `/v3/projects/5d9e68dd66694bfc9416ea893b533ba9/users/89f28574adc9444089ee7f7a1081681b/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:57.988280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1311/2641] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:57 2022] PUT /identity/v3/projects/3ccbbaaf9a874676850c7ba33b832bd0/users/bf77fcc833374b6fa165457bcf647648/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:57.999239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9e68dd66694bfc9416ea893b533ba9, user_id=89f28574adc9444089ee7f7a1081681b, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.002732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815afa5d-f8d6-4273-a64a-82d0b872aa04 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.002732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815afa5d-f8d6-4273-a64a-82d0b872aa04 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.002732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815afa5d-f8d6-4273-a64a-82d0b872aa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.003013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.032341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-815afa5d-f8d6-4273-a64a-82d0b872aa04 None None] MFA Rules not processed for user `bf77fcc833374b6fa165457bcf647648`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:58.033662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-815afa5d-f8d6-4273-a64a-82d0b872aa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.144060 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1331/2642] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:57 2022] PUT /identity/v3/projects/5d9e68dd66694bfc9416ea893b533ba9/users/89f28574adc9444089ee7f7a1081681b/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:08:58.145476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.148883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.219573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09550c3-af69-41af-ab26-ed5153fa24bd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:58.224569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:58.225198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.226087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.226400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] PATH_INFO: `/v3/users/9135eae5264541b49d5bb6507bc81f3d` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.227675 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9135eae5264541b49d5bb6507bc81f3d)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.229644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.233163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1312/2643] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:58.234082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815afa5d-f8d6-4273-a64a-82d0b872aa04 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.300500 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815afa5d-f8d6-4273-a64a-82d0b872aa04 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:58.305510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:58.306187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.306469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.306773 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.307532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.311514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.321381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1313/2644] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:08:58.322427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.324547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.406093 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cc2f0-4d48-487b-8aee-4be6d5f7f950 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:58.409891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:58.410502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.410774 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.411039 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.470587 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.471946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.481838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1314/2645] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:58.484449 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.495392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45800ef-2420-4ed0-b824-23fa773ac7fb service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:58.500325 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfe940-1cc9-41fb-8b4d-1e9241244ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:58.500956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfe940-1cc9-41fb-8b4d-1e9241244ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.501243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfe940-1cc9-41fb-8b4d-1e9241244ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.501515 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfe940-1cc9-41fb-8b4d-1e9241244ee1 admin admin] PATH_INFO: `/v3/projects/5d9e68dd66694bfc9416ea893b533ba9/users/89f28574adc9444089ee7f7a1081681b/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.503795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfe940-1cc9-41fb-8b4d-1e9241244ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9e68dd66694bfc9416ea893b533ba9, user_id=89f28574adc9444089ee7f7a1081681b, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.506571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfe940-1cc9-41fb-8b4d-1e9241244ee1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.519521 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1315/2646] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:08:58 2022] PUT /identity/v3/projects/5d9e68dd66694bfc9416ea893b533ba9/users/89f28574adc9444089ee7f7a1081681b/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:58.530615 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.531013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.531275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.559752 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] MFA Rules not processed for user `89f28574adc9444089ee7f7a1081681b`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:58.560984 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.614371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9135eae5264541b49d5bb6507bc81f3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.622313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9135eae5264541b49d5bb6507bc81f3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.623159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9135eae5264541b49d5bb6507bc81f3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.623855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9135eae5264541b49d5bb6507bc81f3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.634318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1316/2647] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:58.645688 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.647083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.706234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687ae13-1258-4bb9-893c-24b6c1ed4ba5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:58.710967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103a2cc-8388-49d9-991f-19115b3b7a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:58.711728 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103a2cc-8388-49d9-991f-19115b3b7a8c service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.712164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103a2cc-8388-49d9-991f-19115b3b7a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.712555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103a2cc-8388-49d9-991f-19115b3b7a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.714675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d103a2cc-8388-49d9-991f-19115b3b7a8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.727684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103a2cc-8388-49d9-991f-19115b3b7a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:58.729053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103a2cc-8388-49d9-991f-19115b3b7a8c service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:58.739167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1317/2648] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:08:58.821385 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9135eae5264541b49d5bb6507bc81f3d'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:58.841399 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1332/2649] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:08:58 2022] DELETE /identity/v3/users/9135eae5264541b49d5bb6507bc81f3d => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:08:58.846266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:58.846561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:58.846836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:58.865956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:08:58.867217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:58.934678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1318/2650] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:08:58.945222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:58.946785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.011680 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc303e48-d22d-446c-a492-f98544189345 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:59.015357 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:59.016001 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:59.016285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:59.016555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:59.025950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:59.027392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:59.037057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1333/2651] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:59.164136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:59.164136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.216698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dcda3-34a9-43c2-aea6-caa23a40b80b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:59.221088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdf505-a2cf-4398-9823-b93a9e84d033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:59.221088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdf505-a2cf-4398-9823-b93a9e84d033 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:59.221319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdf505-a2cf-4398-9823-b93a9e84d033 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:59.221540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdf505-a2cf-4398-9823-b93a9e84d033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:59.223317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-91cdf505-a2cf-4398-9823-b93a9e84d033 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.264091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:59.297692 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdf505-a2cf-4398-9823-b93a9e84d033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:59.299197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdf505-a2cf-4398-9823-b93a9e84d033 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:59.309695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1319/2652] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:08:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:08:59.611871 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:08:59.613428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:08:59.627554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71473b6f-f448-4082-8204-051cb75d1376 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:08:59.633186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:08:59.633882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:08:59.634168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:08:59.634441 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] PATH_INFO: `/v3/projects/214bbb12729e47c5876c4ed7dd435672` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:08:59.635715 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] RBAC: Authorizing `identity:delete_project(project_id=214bbb12729e47c5876c4ed7dd435672)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:08:59.637673 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:08:59.760279 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:08:59.778539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '214bbb12729e47c5876c4ed7dd435672', 'request_id': 'req-945154ab-fed4-4c4e-9b71-ca729f24f92b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:59.784950 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '214bbb12729e47c5876c4ed7dd435672', 'request_id': 'req-945154ab-fed4-4c4e-9b71-ca729f24f92b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:08:59.792285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '214bbb12729e47c5876c4ed7dd435672', 'request_id': 'req-945154ab-fed4-4c4e-9b71-ca729f24f92b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.022029 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] The token cache is being invalidate because project 214bbb12729e47c5876c4ed7dd435672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:00.022859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.023552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-945154ab-fed4-4c4e-9b71-ca729f24f92b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:00.025103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1334/2653] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:08:59 2022] DELETE /identity/v3/projects/214bbb12729e47c5876c4ed7dd435672 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:00.030329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301ad52-ca23-4b1c-a80f-5a048d71ed8c None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.030811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301ad52-ca23-4b1c-a80f-5a048d71ed8c None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.030943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301ad52-ca23-4b1c-a80f-5a048d71ed8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.050989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f301ad52-ca23-4b1c-a80f-5a048d71ed8c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:00.052188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f301ad52-ca23-4b1c-a80f-5a048d71ed8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.111353 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1320/2654] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:00.122174 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4508-c369-48a3-b232-bff642669a54 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.122538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4508-c369-48a3-b232-bff642669a54 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.122817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4508-c369-48a3-b232-bff642669a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.153037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-05df4508-c369-48a3-b232-bff642669a54 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:00.153037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-05df4508-c369-48a3-b232-bff642669a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:00.158912 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1335/2655] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:00.165086 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301ad52-ca23-4b1c-a80f-5a048d71ed8c None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.181968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301ad52-ca23-4b1c-a80f-5a048d71ed8c None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.187510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.188119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.188392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.188660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.189309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:00.189608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.191874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.204616 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1321/2656] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:00.208893 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df4508-c369-48a3-b232-bff642669a54 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.218702 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df4508-c369-48a3-b232-bff642669a54 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.223365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.223977 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.224251 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.224521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.225156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.227106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.389342 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1336/2657] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:00.394438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.405191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23138f-ed49-4b88-91b0-0fe7f4ca31ea admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.462493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.463097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.463379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.463655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.464381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.466373 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.718900 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1322/2658] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/users => generated 292 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:00.723390 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.735956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e2ee8-4a52-4e4a-b46f-a719ff5118ca admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.742724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.743363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.743646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.743923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.744624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.746391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.758410 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1337/2659] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:00.762839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.776186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f3e20-0b09-421f-a82f-400384f4354d admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.781245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135293e-c62d-4b13-90c4-49a3c0d32bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.781903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135293e-c62d-4b13-90c4-49a3c0d32bdf admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.782356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135293e-c62d-4b13-90c4-49a3c0d32bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.782512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135293e-c62d-4b13-90c4-49a3c0d32bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.783140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135293e-c62d-4b13-90c4-49a3c0d32bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.784775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135293e-c62d-4b13-90c4-49a3c0d32bdf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.801097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1323/2660] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:00.802755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:00.815041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4029b-b3d1-4cfa-9f5e-a57c6fb14c7b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:00.820402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:00.821120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.821400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.821677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] PATH_INFO: `/v3/projects/4359f43bdc044cb99d2ef46b8f05d194/users/47952183ab7a4ee9b19e40e25b75ea1c/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.838852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4359f43bdc044cb99d2ef46b8f05d194, user_id=47952183ab7a4ee9b19e40e25b75ea1c, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:00.844207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:00.924464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1338/2661] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:00 2022] PUT /identity/v3/projects/4359f43bdc044cb99d2ef46b8f05d194/users/47952183ab7a4ee9b19e40e25b75ea1c/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:00.934759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fea2bd-dfa7-4cff-9f22-d0fb91b84b0b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:00.935046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fea2bd-dfa7-4cff-9f22-d0fb91b84b0b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:00.935329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fea2bd-dfa7-4cff-9f22-d0fb91b84b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:00.964176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-76fea2bd-dfa7-4cff-9f22-d0fb91b84b0b None None] MFA Rules not processed for user `47952183ab7a4ee9b19e40e25b75ea1c`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:00.965433 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-76fea2bd-dfa7-4cff-9f22-d0fb91b84b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:01.033648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1324/2662] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:01.038305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.101081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f1651-5e5d-4c72-b23f-2fb34a736422 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.106113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.106748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.107027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.107302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.107951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.109903 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.258551 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1339/2663] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:01 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:01.263418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fea2bd-dfa7-4cff-9f22-d0fb91b84b0b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.274478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fea2bd-dfa7-4cff-9f22-d0fb91b84b0b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.333109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.333726 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.334030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.334306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.334976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.336887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.525276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1325/2664] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:01 2022] POST /identity/v3/users => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:01.529164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.539766 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa48bc-b5f7-46be-ae40-89d1769f2f28 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.544309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.544904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.545178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.545454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.546160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.547741 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.558662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1340/2665] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:01.562878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.573536 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6987e-ec24-4749-87e4-3e8c49af1f3f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.578293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.578952 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.579230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.579501 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] PATH_INFO: `/v3/projects/d271d6a647504eb18dcb8d7695978ace/users/cf77dcd14e0c4a6fa70644ad9fc3d4f7/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.594081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d271d6a647504eb18dcb8d7695978ace, user_id=cf77dcd14e0c4a6fa70644ad9fc3d4f7, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.596863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.816976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1326/2666] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:01 2022] PUT /identity/v3/projects/d271d6a647504eb18dcb8d7695978ace/users/cf77dcd14e0c4a6fa70644ad9fc3d4f7/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:01.822324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.838236 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bc55-ea93-46e0-a9aa-1ef50096d3ac admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.868065 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:01.901194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68328b-0070-4c6b-b645-d9e32cd8df25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.901883 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68328b-0070-4c6b-b645-d9e32cd8df25 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.902245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68328b-0070-4c6b-b645-d9e32cd8df25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.902442 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68328b-0070-4c6b-b645-d9e32cd8df25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.903106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68328b-0070-4c6b-b645-d9e32cd8df25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.904783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68328b-0070-4c6b-b645-d9e32cd8df25 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.916525 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1341/2667] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:01.922874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:01.934425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9438c-e1fb-4b5d-b43b-fe36cf7d3d1a admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:01.939193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:01.941759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.941759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.941759 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] PATH_INFO: `/v3/projects/d271d6a647504eb18dcb8d7695978ace/users/cf77dcd14e0c4a6fa70644ad9fc3d4f7/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.942729 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d271d6a647504eb18dcb8d7695978ace, user_id=cf77dcd14e0c4a6fa70644ad9fc3d4f7, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:01.945462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:01.956388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1327/2668] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:01 2022] PUT /identity/v3/projects/d271d6a647504eb18dcb8d7695978ace/users/cf77dcd14e0c4a6fa70644ad9fc3d4f7/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:01.967647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:01.967931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:01.968207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:01.996206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] MFA Rules not processed for user `cf77dcd14e0c4a6fa70644ad9fc3d4f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:01.997451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:02.068567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1342/2669] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:02.080880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:02.082632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:02.144664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b223fd5-a0aa-4353-bc16-fbe794f43e0e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:02.148935 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:02.149540 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:02.149848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:02.150123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:02.159189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:02.160613 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:02.171680 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1328/2670] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:02.386274 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:02.389341 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:02.404814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcc91b-d7bd-424a-a970-684a6b510f2a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:02.408626 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac530d9-9015-4b35-ae33-4848bdb10b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:02.409243 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac530d9-9015-4b35-ae33-4848bdb10b27 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:02.409514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac530d9-9015-4b35-ae33-4848bdb10b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:02.409814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac530d9-9015-4b35-ae33-4848bdb10b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:02.410707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:02.467710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac530d9-9015-4b35-ae33-4848bdb10b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:02.469144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac530d9-9015-4b35-ae33-4848bdb10b27 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:02.478645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1343/2671] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:03.516874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:03.594318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc278ad-1da5-4488-b722-139645684d70 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:03.599318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:03.603384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:03.603384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:03.603384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] PATH_INFO: `/v3/users/47952183ab7a4ee9b19e40e25b75ea1c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:03.603384 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] RBAC: Authorizing `identity:delete_user(user_id=47952183ab7a4ee9b19e40e25b75ea1c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:03.604062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:03.892915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47952183ab7a4ee9b19e40e25b75ea1c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:03.900845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47952183ab7a4ee9b19e40e25b75ea1c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:03.988036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47952183ab7a4ee9b19e40e25b75ea1c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:03.989231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47952183ab7a4ee9b19e40e25b75ea1c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:04.004657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47952183ab7a4ee9b19e40e25b75ea1c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:04.010947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1329/2672] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:03 2022] DELETE /identity/v3/users/47952183ab7a4ee9b19e40e25b75ea1c => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:04.015517 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:04.016017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:04.016131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:04.035829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:04.037101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:04.108041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1344/2673] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:04.119061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:04.120494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:04.180649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f9ba0-2998-448e-bcdc-b067579cf68d admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:04.184399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:04.185010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:04.185372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:04.185625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:04.196380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:04.197892 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:04.207939 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1330/2674] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:09:04.645001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:04.911282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:04.912727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:04.926584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf1b67-1ec6-4da3-abfc-12905459a0fd None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:04.938149 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:04.938853 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:04.939131 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:04.939608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] PATH_INFO: `/v3/projects/4359f43bdc044cb99d2ef46b8f05d194` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:04.940962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4359f43bdc044cb99d2ef46b8f05d194)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:04.943115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:05.080637 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4359f43bdc044cb99d2ef46b8f05d194', 'request_id': 'req-e5563bf2-5cd4-420e-8751-73038aea475a'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.086771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4359f43bdc044cb99d2ef46b8f05d194', 'request_id': 'req-e5563bf2-5cd4-420e-8751-73038aea475a'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.092531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4359f43bdc044cb99d2ef46b8f05d194', 'request_id': 'req-e5563bf2-5cd4-420e-8751-73038aea475a'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.187649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:05.278509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] The token cache is being invalidate because project 4359f43bdc044cb99d2ef46b8f05d194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:05.278951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.279636 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.281130 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1345/2675] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:04 2022] DELETE /identity/v3/projects/4359f43bdc044cb99d2ef46b8f05d194 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:05.285047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:05.295338 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20eb7-6cf0-4bc5-905d-cad9752c9cd4 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:05.353199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:05.353875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:05.354150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:05.354420 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] PATH_INFO: `/v3/users/cf77dcd14e0c4a6fa70644ad9fc3d4f7` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:05.355561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf77dcd14e0c4a6fa70644ad9fc3d4f7)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:05.357547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:05.819685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf77dcd14e0c4a6fa70644ad9fc3d4f7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.827089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf77dcd14e0c4a6fa70644ad9fc3d4f7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.901585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf77dcd14e0c4a6fa70644ad9fc3d4f7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.902499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf77dcd14e0c4a6fa70644ad9fc3d4f7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.916262 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf77dcd14e0c4a6fa70644ad9fc3d4f7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:05.922727 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1331/2676] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:05 2022] DELETE /identity/v3/users/cf77dcd14e0c4a6fa70644ad9fc3d4f7 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:06.233431 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:06.235105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:06.252238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563bf2-5cd4-420e-8751-73038aea475a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:06.323586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:06.324239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:06.324514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:06.335512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] PATH_INFO: `/v3/users/bf77fcc833374b6fa165457bcf647648` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:06.335512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf77fcc833374b6fa165457bcf647648)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:06.335512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:06.579547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:06.581097 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:06.595001 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7373c-dc88-4991-abc9-8c9315b628e4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:06.601856 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:06.602876 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:06.603156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:06.603561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] PATH_INFO: `/v3/projects/d271d6a647504eb18dcb8d7695978ace` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:06.605060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d271d6a647504eb18dcb8d7695978ace)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:06.607200 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:06.678167 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf77fcc833374b6fa165457bcf647648'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.685672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf77fcc833374b6fa165457bcf647648'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.686491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf77fcc833374b6fa165457bcf647648'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.687226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf77fcc833374b6fa165457bcf647648'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.819958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd271d6a647504eb18dcb8d7695978ace', 'request_id': 'req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.821582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-b3737daa-7860-4d91-b996-0adabcaf2da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf77fcc833374b6fa165457bcf647648'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.825916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd271d6a647504eb18dcb8d7695978ace', 'request_id': 'req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:06.841957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1346/2677] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:06 2022] DELETE /identity/v3/users/bf77fcc833374b6fa165457bcf647648 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:06.846563 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:06.846855 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:06.847129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:06.867660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:06.869240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:06.965025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1347/2678] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:06.976946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:06.979027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:06.993380 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd271d6a647504eb18dcb8d7695978ace', 'request_id': 'req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:07.007400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] The token cache is being invalidate because project d271d6a647504eb18dcb8d7695978ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:07.007706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:07.008415 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:07.010299 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1332/2679] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:06 2022] DELETE /identity/v3/projects/d271d6a647504eb18dcb8d7695978ace => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:07.064844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab099fb-7efe-40e3-8c38-51a83e0f21fb None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:07.069699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25456a61-30ff-4108-afe2-554bebf2c676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:07.070432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25456a61-30ff-4108-afe2-554bebf2c676 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:07.070656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25456a61-30ff-4108-afe2-554bebf2c676 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:07.070935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25456a61-30ff-4108-afe2-554bebf2c676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:07.073199 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-25456a61-30ff-4108-afe2-554bebf2c676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:07.088447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25456a61-30ff-4108-afe2-554bebf2c676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:07.091404 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25456a61-30ff-4108-afe2-554bebf2c676 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:07.107646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1348/2680] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:07.710869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:07.712769 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:07.795886 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b0e37-53b9-414f-ab52-b9c2ecfbc93c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:07.800847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:07.801509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:07.801811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:07.802094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] PATH_INFO: `/v3/projects/3ccbbaaf9a874676850c7ba33b832bd0` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:07.803302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ccbbaaf9a874676850c7ba33b832bd0)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:07.805210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:07.930223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ccbbaaf9a874676850c7ba33b832bd0', 'request_id': 'req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:07.935673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ccbbaaf9a874676850c7ba33b832bd0', 'request_id': 'req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.010247 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:08.044417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ccbbaaf9a874676850c7ba33b832bd0', 'request_id': 'req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.057340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] The token cache is being invalidate because project 3ccbbaaf9a874676850c7ba33b832bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:08.057636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.058346 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-1d4a4d78-12aa-4979-988d-af5f7f9b4d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:08.059949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1333/2681] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:07 2022] DELETE /identity/v3/projects/3ccbbaaf9a874676850c7ba33b832bd0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:08.065250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f84fa-5ba4-49e4-98e8-417c9f859be6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.065881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f84fa-5ba4-49e4-98e8-417c9f859be6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.065881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f84fa-5ba4-49e4-98e8-417c9f859be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.086843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-366f84fa-5ba4-49e4-98e8-417c9f859be6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:08.088073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-366f84fa-5ba4-49e4-98e8-417c9f859be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.160696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1349/2682] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:08.168916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294c19-3411-435f-9102-34a39dca9fdd None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.169197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294c19-3411-435f-9102-34a39dca9fdd None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.169468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294c19-3411-435f-9102-34a39dca9fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.188674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-32294c19-3411-435f-9102-34a39dca9fdd None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:08.189918 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-32294c19-3411-435f-9102-34a39dca9fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.197709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1334/2683] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:08.201955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f84fa-5ba4-49e4-98e8-417c9f859be6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.212233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f84fa-5ba4-49e4-98e8-417c9f859be6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.216856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.217451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.217727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.218026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.218670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:08.218972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.220569 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.233088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1350/2684] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:08.237450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294c19-3411-435f-9102-34a39dca9fdd None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.246869 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294c19-3411-435f-9102-34a39dca9fdd None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.251795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.252414 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.252697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.252999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.253633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.255842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.308106 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:08.403636 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1335/2685] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:08 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:08.405698 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.416658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448d86-1947-42b8-b6bb-88fa229338b1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.475069 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.475682 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.475983 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.476256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.477002 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.479118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.616958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1351/2686] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:08 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:08.621017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.632282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642e990-6127-491a-96a7-6e90e1268cba admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.637033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.637654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.638681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.639201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.640559 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.643011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.654549 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1336/2687] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:08.659232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.669804 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177364f4-9f42-4bb5-bb00-85017d4e97af admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.674911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2860d-ff9e-4794-82de-05954f399e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.675568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2860d-ff9e-4794-82de-05954f399e33 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.675848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2860d-ff9e-4794-82de-05954f399e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.676222 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2860d-ff9e-4794-82de-05954f399e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.676945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2860d-ff9e-4794-82de-05954f399e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.679267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2860d-ff9e-4794-82de-05954f399e33 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.690834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1352/2688] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:08.695601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.705817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca873a-ccc4-49f2-a4a6-fba21b7d50db admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.710612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.711327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.711644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.711891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] PATH_INFO: `/v3/projects/035b87bdd94646d0aed7727e587ab40f/users/35383b224a5a46de87c41b49fe1bd037/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.726944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035b87bdd94646d0aed7727e587ab40f, user_id=35383b224a5a46de87c41b49fe1bd037, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:08.729669 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:08.824590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1337/2689] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:08 2022] PUT /identity/v3/projects/035b87bdd94646d0aed7727e587ab40f/users/35383b224a5a46de87c41b49fe1bd037/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:08.829795 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b8f0d-7f6d-4909-afae-df2be3f84fd8 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.830083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b8f0d-7f6d-4909-afae-df2be3f84fd8 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.830421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b8f0d-7f6d-4909-afae-df2be3f84fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:08.851531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d83b8f0d-7f6d-4909-afae-df2be3f84fd8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:08.852769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d83b8f0d-7f6d-4909-afae-df2be3f84fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.920028 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1353/2690] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:08.931206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:08.933211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:08.990166 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302afc1-0e2f-48b8-b8ba-1dd7ebaa8b59 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:08.994378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:08.994985 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:08.995263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:08.995542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:09.004539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:09.006013 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:09.015976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1338/2691] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:10.028450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:10.421639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:10.767516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77302-9b17-48a8-8df3-0d46f2c118ed None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:10.767805 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77302-9b17-48a8-8df3-0d46f2c118ed None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:10.768355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77302-9b17-48a8-8df3-0d46f2c118ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:10.800455 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-44f77302-9b17-48a8-8df3-0d46f2c118ed None None] MFA Rules not processed for user `35383b224a5a46de87c41b49fe1bd037`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:10.801923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-44f77302-9b17-48a8-8df3-0d46f2c118ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:10.876923 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1354/2692] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:10.903158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:10.906237 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:10.976051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362c940-a0d6-4a7c-8870-bc1e5790077a service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:10.976821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:10.977456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:10.978003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:10.978269 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:10.987620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:10.989234 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:11.008976 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1339/2693] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:12.604549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:12.879232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:13.030787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d3f2-9795-4975-895c-f315a5ecd78b None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:13.031414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d3f2-9795-4975-895c-f315a5ecd78b None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:13.031819 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d3f2-9795-4975-895c-f315a5ecd78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:13.053937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-09c3d3f2-9795-4975-895c-f315a5ecd78b None None] MFA Rules not processed for user `35383b224a5a46de87c41b49fe1bd037`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:13.055315 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-09c3d3f2-9795-4975-895c-f315a5ecd78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:13.062642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1355/2694] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:14.444733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:14.893872 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:14.895847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:14.916036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c8a27-5ed0-4132-b0e9-e7573712d3ef service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:14.921518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:14.922044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:14.922342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:14.922571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] PATH_INFO: `/v3/users/3e8d6662fc0b4f35bdc7ea178d7eb0ba` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:14.923841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8d6662fc0b4f35bdc7ea178d7eb0ba)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:14.925848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:15.168991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8d6662fc0b4f35bdc7ea178d7eb0ba'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.176779 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8d6662fc0b4f35bdc7ea178d7eb0ba'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.330194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8d6662fc0b4f35bdc7ea178d7eb0ba'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.330949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8d6662fc0b4f35bdc7ea178d7eb0ba'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.350958 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8d6662fc0b4f35bdc7ea178d7eb0ba'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:15.351644 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1340/2695] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:14 2022] DELETE /identity/v3/users/3e8d6662fc0b4f35bdc7ea178d7eb0ba => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:15.353656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:15.356354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:15.356651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:15.356916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:15.376047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:15.377277 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:15.443271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1356/2696] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:15.455599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:15.457499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:15.524965 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb02481-91bb-4f21-b55e-4564d5162d62 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:15.529025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:15.529654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:15.529967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:15.530246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:15.540877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:15.542359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:15.552960 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1341/2697] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:16.067573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:16.152395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:16.156112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:16.188049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ae608-d78d-4c9c-8b21-bb2cdf349639 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:16.193917 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:16.194881 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:16.195138 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:16.195656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] PATH_INFO: `/v3/projects/7a48453b4f0e48e5be4d83ced4691e24` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:16.197971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a48453b4f0e48e5be4d83ced4691e24)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:16.200210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:16.373318 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a48453b4f0e48e5be4d83ced4691e24', 'request_id': 'req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.381151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a48453b4f0e48e5be4d83ced4691e24', 'request_id': 'req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.387568 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a48453b4f0e48e5be4d83ced4691e24', 'request_id': 'req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.536110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] The token cache is being invalidate because project 7a48453b4f0e48e5be4d83ced4691e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:16.536448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.537678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:16.539734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1357/2698] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:16 2022] DELETE /identity/v3/projects/7a48453b4f0e48e5be4d83ced4691e24 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:16.544416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:16.555065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a4200-b4ef-41ca-b9b5-e0973859efcd service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:16.611137 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:16.611798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:16.612077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:16.612359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] PATH_INFO: `/v3/users/89f28574adc9444089ee7f7a1081681b` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:16.613662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f28574adc9444089ee7f7a1081681b)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:16.616216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:17.043732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f28574adc9444089ee7f7a1081681b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:17.052350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f28574adc9444089ee7f7a1081681b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:17.158917 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:17.204968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f28574adc9444089ee7f7a1081681b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:17.205639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f28574adc9444089ee7f7a1081681b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:17.221029 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f28574adc9444089ee7f7a1081681b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:17.228128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1342/2699] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:16 2022] DELETE /identity/v3/users/89f28574adc9444089ee7f7a1081681b => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:17.444669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:17.446133 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:17.542773 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb4dea-d88a-4f10-b2d1-d3c1dfb503a9 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:17.612866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebba77-bdcd-4193-9aae-9118aad405bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:17.613792 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebba77-bdcd-4193-9aae-9118aad405bb service placement] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:17.614777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebba77-bdcd-4193-9aae-9118aad405bb service placement] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:17.615095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebba77-bdcd-4193-9aae-9118aad405bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:17.618207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-02ebba77-bdcd-4193-9aae-9118aad405bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:17.631646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebba77-bdcd-4193-9aae-9118aad405bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:17.633285 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebba77-bdcd-4193-9aae-9118aad405bb service placement] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:17.642272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:17.648649 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1358/2700] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:17.873063 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:17.875334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:17.951753 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524fedb-b3fc-4521-85ae-fe263c703740 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:17.956306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:17.956945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:17.957221 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:17.957495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] PATH_INFO: `/v3/projects/5d9e68dd66694bfc9416ea893b533ba9` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:17.958656 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9e68dd66694bfc9416ea893b533ba9)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:17.960584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:18.148002 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9e68dd66694bfc9416ea893b533ba9', 'request_id': 'req-a86fe593-d246-4454-af5c-14b91b884a37'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:18.157312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9e68dd66694bfc9416ea893b533ba9', 'request_id': 'req-a86fe593-d246-4454-af5c-14b91b884a37'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:18.267532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9e68dd66694bfc9416ea893b533ba9', 'request_id': 'req-a86fe593-d246-4454-af5c-14b91b884a37'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:18.281846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] The token cache is being invalidate because project 5d9e68dd66694bfc9416ea893b533ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:18.282138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:18.282931 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-a86fe593-d246-4454-af5c-14b91b884a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:18.284436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1343/2701] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:17 2022] DELETE /identity/v3/projects/5d9e68dd66694bfc9416ea893b533ba9 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:18.289363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7751fc-409e-43b8-812b-f2a2453306d5 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.289641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7751fc-409e-43b8-812b-f2a2453306d5 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.289932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7751fc-409e-43b8-812b-f2a2453306d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.309922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-be7751fc-409e-43b8-812b-f2a2453306d5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:18.311304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-be7751fc-409e-43b8-812b-f2a2453306d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:18.358760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:18.374429 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1359/2702] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:18.383186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0c1b7-09fa-4f0a-a7af-1314e5f9e1b5 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.384055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0c1b7-09fa-4f0a-a7af-1314e5f9e1b5 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.384399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0c1b7-09fa-4f0a-a7af-1314e5f9e1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.405140 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-37b0c1b7-09fa-4f0a-a7af-1314e5f9e1b5 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:18.406485 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-37b0c1b7-09fa-4f0a-a7af-1314e5f9e1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:18.416367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1344/2703] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:18.421661 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7751fc-409e-43b8-812b-f2a2453306d5 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:18.440502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7751fc-409e-43b8-812b-f2a2453306d5 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:18.445927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:18.450204 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.450560 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.450851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.452039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:18.452039 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:18.453483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:18.467601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1360/2704] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:18.471860 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0c1b7-09fa-4f0a-a7af-1314e5f9e1b5 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:18.481866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0c1b7-09fa-4f0a-a7af-1314e5f9e1b5 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:18.486511 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:18.487107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.487386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.487660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.488340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:18.490404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:18.663906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1345/2705] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:18 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:18.668839 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:18.680644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f781b-52d8-49d8-835d-7bfa7a579f30 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:18.743818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:18.744475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.744751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.745026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.745757 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:18.747760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:18.913207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1361/2706] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:18 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:18.917567 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:18.928577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83125-0c65-47d1-af11-77f16061dc96 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:18.933486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:18.934124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.934454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.934738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.935422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:18.937110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:18.949158 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1346/2707] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:18.953050 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:18.963275 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f8687-3aa6-4052-a7d7-20dbf999abfd admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:18.967957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c792125-c632-4549-91d2-0e9e86c06310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:18.968562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c792125-c632-4549-91d2-0e9e86c06310 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:18.968842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c792125-c632-4549-91d2-0e9e86c06310 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:18.969115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c792125-c632-4549-91d2-0e9e86c06310 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:18.969785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c792125-c632-4549-91d2-0e9e86c06310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:18.971421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c792125-c632-4549-91d2-0e9e86c06310 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:18.985000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1362/2708] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:18.991121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:19.002597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae30ec-d785-4d77-94c5-14650566adda admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:19.009801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:19.010907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:19.011358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:19.011804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] PATH_INFO: `/v3/projects/8c4163e503d64faf9623e454ea45f95c/users/603c2f79645b46b8bb27d988e965b94b/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:19.029342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4163e503d64faf9623e454ea45f95c, user_id=603c2f79645b46b8bb27d988e965b94b, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:19.032175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:19.133051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1347/2709] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:18 2022] PUT /identity/v3/projects/8c4163e503d64faf9623e454ea45f95c/users/603c2f79645b46b8bb27d988e965b94b/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:19.144913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cb414-ad92-4376-a7dc-54c308706d9f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:19.145188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cb414-ad92-4376-a7dc-54c308706d9f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:19.145459 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cb414-ad92-4376-a7dc-54c308706d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:19.183019 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-9a3cb414-ad92-4376-a7dc-54c308706d9f None None] MFA Rules not processed for user `603c2f79645b46b8bb27d988e965b94b`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:19.184221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-9a3cb414-ad92-4376-a7dc-54c308706d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:19.255612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1363/2710] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:19.260736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:19.262541 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:19.331456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aaee2-9a32-4257-847f-207454766e19 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:19.337696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:19.338377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:19.338657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:19.339015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:19.341256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:19.343429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:19.487737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1348/2711] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:19 2022] POST /identity/v3/projects => generated 363 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:19.491559 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cb414-ad92-4376-a7dc-54c308706d9f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:19.516954 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cb414-ad92-4376-a7dc-54c308706d9f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:19.592329 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:19.593858 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:19.594414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:19.595422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:19.601595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:19.601595 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:19.633439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:19.736776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1364/2712] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:19 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:19.740894 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:19.751069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad913f5a-e603-47bc-9215-bd4838f7f23f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:19.757186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:19.763723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:19.763723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:19.763723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:19.763723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:19.763723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:19.774801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1349/2713] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:09:19.779216 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:19.789438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc15e8-6405-41b2-9e45-78c33214deb2 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:19.794182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:19.794833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:19.795113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:19.795391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] PATH_INFO: `/v3/projects/226634b624804d33b03d1a4c2fcc795d/users/ced6255667814aeb91b14d9d46735309/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:19.810586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=226634b624804d33b03d1a4c2fcc795d, user_id=ced6255667814aeb91b14d9d46735309, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:19.813268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:19.939308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1365/2714] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:19 2022] PUT /identity/v3/projects/226634b624804d33b03d1a4c2fcc795d/users/ced6255667814aeb91b14d9d46735309/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:19.944255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:19.958280 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150d914-d9b4-4994-ada0-a6761a01ea79 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:20.023742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d5b5e-15cb-4c77-b33b-5b4e00833541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:20.028825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d5b5e-15cb-4c77-b33b-5b4e00833541 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:20.028825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d5b5e-15cb-4c77-b33b-5b4e00833541 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:20.028825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d5b5e-15cb-4c77-b33b-5b4e00833541 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:20.028825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d5b5e-15cb-4c77-b33b-5b4e00833541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:20.028825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d5b5e-15cb-4c77-b33b-5b4e00833541 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:20.038058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1350/2715] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:20.042452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:20.053622 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b948a-09ae-4769-8b29-10137aec4cfe admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:20.059547 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:20.060291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:20.060575 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:20.060918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] PATH_INFO: `/v3/projects/226634b624804d33b03d1a4c2fcc795d/users/ced6255667814aeb91b14d9d46735309/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:20.063653 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226634b624804d33b03d1a4c2fcc795d, user_id=ced6255667814aeb91b14d9d46735309, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:20.070052 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:20.087594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1366/2716] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:20 2022] PUT /identity/v3/projects/226634b624804d33b03d1a4c2fcc795d/users/ced6255667814aeb91b14d9d46735309/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:20.109182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064474-22b8-4ba3-91c7-17117efd6b03 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:20.109182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064474-22b8-4ba3-91c7-17117efd6b03 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:20.109182 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064474-22b8-4ba3-91c7-17117efd6b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:20.131796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7c064474-22b8-4ba3-91c7-17117efd6b03 None None] MFA Rules not processed for user `ced6255667814aeb91b14d9d46735309`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:20.133028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7c064474-22b8-4ba3-91c7-17117efd6b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:20.211648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1351/2717] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:20.215658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:20.223324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:20.226714 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:20.307281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7737e1-59e3-427d-aaae-370662ab49c4 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:20.311054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:20.311725 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:20.311941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:20.312202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:20.322354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:20.324048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:20.334424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1367/2718] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:20.666747 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:20.675620 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c064474-22b8-4ba3-91c7-17117efd6b03 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:20.687283 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c064474-22b8-4ba3-91c7-17117efd6b03 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:20.691017 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88af48-ee14-44cf-b471-e39d7f1ccc89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:20.691596 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88af48-ee14-44cf-b471-e39d7f1ccc89 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:20.691878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88af48-ee14-44cf-b471-e39d7f1ccc89 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:20.692161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88af48-ee14-44cf-b471-e39d7f1ccc89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:20.754713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88af48-ee14-44cf-b471-e39d7f1ccc89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:20.756857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88af48-ee14-44cf-b471-e39d7f1ccc89 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:20.766789 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1352/2719] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:20.935891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:21.524513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:21.930460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:22.192944 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:22.404388 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:22.826693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:23.032717 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:23.198894 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:23.257010 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:23.467726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:23.570803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:23.661175 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:23.930802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:24.273869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:24.621991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:24.706979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:25.250366 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:25.335972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:25.547388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:25.718639 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:35.052005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:35.134623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8201b4-b760-413c-aa6f-983a47474680 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:35.139837 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:35.140665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:35.140946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:35.141250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] PATH_INFO: `/v3/users/603c2f79645b46b8bb27d988e965b94b` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:35.142528 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603c2f79645b46b8bb27d988e965b94b)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:35.144521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:35.425589 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603c2f79645b46b8bb27d988e965b94b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:35.433737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603c2f79645b46b8bb27d988e965b94b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:35.434632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603c2f79645b46b8bb27d988e965b94b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:35.435460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603c2f79645b46b8bb27d988e965b94b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:35.598562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603c2f79645b46b8bb27d988e965b94b'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:35.619475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1368/2720] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:35 2022] DELETE /identity/v3/users/603c2f79645b46b8bb27d988e965b94b => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:35.624096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:35.624890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:35.624890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:35.645094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:35.646851 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:35.715367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1353/2721] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:35.725286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:35.726777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:35.778774 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:35.791798 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494592ef-e3f2-4a39-92de-9b0997950ea4 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:35.796878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:35.797546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:35.797862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:35.798141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:35.810224 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:35.812235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:35.823585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1369/2722] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:36.421715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:36.421715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:36.437110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599a80a-b744-4f44-8e55-e5aaf2fcd9ff None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:36.444768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:36.445425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:36.445661 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:36.446127 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] PATH_INFO: `/v3/projects/8c4163e503d64faf9623e454ea45f95c` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:36.447903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c4163e503d64faf9623e454ea45f95c)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:36.449990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:36.467928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:36.649211 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c4163e503d64faf9623e454ea45f95c', 'request_id': 'req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:36.654495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c4163e503d64faf9623e454ea45f95c', 'request_id': 'req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:36.815826 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c4163e503d64faf9623e454ea45f95c', 'request_id': 'req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:36.827806 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] The token cache is being invalidate because project 8c4163e503d64faf9623e454ea45f95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:36.828098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:36.828730 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:36.830250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1354/2723] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:36 2022] DELETE /identity/v3/projects/8c4163e503d64faf9623e454ea45f95c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:36.834692 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:36.844510 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cdb9e-f26c-4a67-9f62-59a94e7641d7 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:36.906190 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:36.906769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:36.907042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:36.907309 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] PATH_INFO: `/v3/users/ced6255667814aeb91b14d9d46735309` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:36.908417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced6255667814aeb91b14d9d46735309)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:36.910451 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:37.379910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced6255667814aeb91b14d9d46735309'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.387269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced6255667814aeb91b14d9d46735309'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.387957 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced6255667814aeb91b14d9d46735309'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.388564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced6255667814aeb91b14d9d46735309'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.452083 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:37.453557 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:37.467213 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec52fc1-56d3-4f12-9b99-5e0979be7ad6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:37.472161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:37.472934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:37.473068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:37.473340 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] PATH_INFO: `/v3/users/7d6131abf44e46a5b7fe076cf4173757` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:37.475009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6131abf44e46a5b7fe076cf4173757)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:37.476995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:37.478384 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-9f47a9af-1a3f-4d8b-95b8-76ff5e3dbbcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced6255667814aeb91b14d9d46735309'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.498860 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1370/2724] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:36 2022] DELETE /identity/v3/users/ced6255667814aeb91b14d9d46735309 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:37.758435 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6131abf44e46a5b7fe076cf4173757'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.765585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6131abf44e46a5b7fe076cf4173757'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.861453 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6131abf44e46a5b7fe076cf4173757'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.862198 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6131abf44e46a5b7fe076cf4173757'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.877945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6131abf44e46a5b7fe076cf4173757'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:37.885193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1355/2725] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:37 2022] DELETE /identity/v3/users/7d6131abf44e46a5b7fe076cf4173757 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:37.889520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:37.892108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:37.892108 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:37.910270 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:37.912197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:37.972290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1371/2726] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:37.997991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:37.997991 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:38.000956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:38.002933 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:38.012771 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:38.018273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1116b-27dc-46c5-9516-c14f3da8411f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:38.024259 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:38.025063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.025501 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.025947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] PATH_INFO: `/v3/projects/226634b624804d33b03d1a4c2fcc795d` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.027382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=226634b624804d33b03d1a4c2fcc795d)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:38.029417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:38.072942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2b999-950f-4bb5-ae46-71d3c14965ea admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:38.076571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d6938-1dfe-45c6-9690-39fea43be70a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:38.082374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d6938-1dfe-45c6-9690-39fea43be70a service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.082374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d6938-1dfe-45c6-9690-39fea43be70a service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.082374 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d6938-1dfe-45c6-9690-39fea43be70a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.086828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d6938-1dfe-45c6-9690-39fea43be70a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:38.088259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d6938-1dfe-45c6-9690-39fea43be70a service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:38.097853 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1356/2727] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:38.244931 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226634b624804d33b03d1a4c2fcc795d', 'request_id': 'req-8092978b-f44e-4e16-8d22-2948455da775'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:38.251326 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226634b624804d33b03d1a4c2fcc795d', 'request_id': 'req-8092978b-f44e-4e16-8d22-2948455da775'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:38.257574 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226634b624804d33b03d1a4c2fcc795d', 'request_id': 'req-8092978b-f44e-4e16-8d22-2948455da775'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:38.428094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] The token cache is being invalidate because project 226634b624804d33b03d1a4c2fcc795d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:38.428485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:38.429177 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8092978b-f44e-4e16-8d22-2948455da775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:38.431435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1372/2728] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:37 2022] DELETE /identity/v3/projects/226634b624804d33b03d1a4c2fcc795d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:38.446505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d04ec-fa7b-4a8e-a9fb-c80b4df67769 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.446505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d04ec-fa7b-4a8e-a9fb-c80b4df67769 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.446505 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d04ec-fa7b-4a8e-a9fb-c80b4df67769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.465456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0a9d04ec-fa7b-4a8e-a9fb-c80b4df67769 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:38.466710 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0a9d04ec-fa7b-4a8e-a9fb-c80b4df67769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:38.480122 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:38.541698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1357/2729] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:38.542331 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28fdd-395c-4cf1-82cb-7f8643f30d55 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.543308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28fdd-395c-4cf1-82cb-7f8643f30d55 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.543308 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28fdd-395c-4cf1-82cb-7f8643f30d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.563532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-89d28fdd-395c-4cf1-82cb-7f8643f30d55 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:38.564838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-89d28fdd-395c-4cf1-82cb-7f8643f30d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:38.573209 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1373/2730] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:38.577719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d04ec-fa7b-4a8e-a9fb-c80b4df67769 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:38.588681 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d04ec-fa7b-4a8e-a9fb-c80b4df67769 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:38.593330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:38.599649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.599649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.599649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.599649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:38.599649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:38.599649 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:38.609427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1358/2731] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:38.614047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d28fdd-395c-4cf1-82cb-7f8643f30d55 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:38.623290 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d28fdd-395c-4cf1-82cb-7f8643f30d55 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:38.628151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:38.628765 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.629071 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.629631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.630386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:38.632598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:38.679786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:38.684348 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:38.696350 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b9dc-98b3-409c-8808-0faa86d18ec4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:38.704780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:38.705419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.705836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.705972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] PATH_INFO: `/v3/projects/5191ebeda1ab4de3901f00fee0a7be3e` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.710973 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5191ebeda1ab4de3901f00fee0a7be3e)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:38.712972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:38.789041 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1374/2732] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:38 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:38.793409 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:38.855918 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236026-a5a8-41b3-8b7d-1af4f7af0da1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:38.860726 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:38.861333 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:38.861612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:38.861913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:38.862549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:38.864511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:38.909026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5191ebeda1ab4de3901f00fee0a7be3e', 'request_id': 'req-e5f40891-efb9-4af7-b99f-5be5b215e929'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:38.916282 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5191ebeda1ab4de3901f00fee0a7be3e', 'request_id': 'req-e5f40891-efb9-4af7-b99f-5be5b215e929'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:39.021852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1375/2733] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:38 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:39.026055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.036047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406674d-fe29-4f20-884d-afd1b32393ee admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.085634 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5191ebeda1ab4de3901f00fee0a7be3e', 'request_id': 'req-e5f40891-efb9-4af7-b99f-5be5b215e929'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:39.089980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:39.110164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.110164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.110164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.110164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.110536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.110614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] The token cache is being invalidate because project 5191ebeda1ab4de3901f00fee0a7be3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:39.110614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:39.110778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.110858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e5f40891-efb9-4af7-b99f-5be5b215e929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:39.110858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1359/2734] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:38 2022] DELETE /identity/v3/projects/5191ebeda1ab4de3901f00fee0a7be3e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:39.111547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826147-b917-41c6-b8d6-ded706cf3af1 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.111834 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826147-b917-41c6-b8d6-ded706cf3af1 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.112274 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826147-b917-41c6-b8d6-ded706cf3af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.121506 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1376/2735] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:09:39.125862 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.127624 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:39.132396 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-a6826147-b917-41c6-b8d6-ded706cf3af1 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:39.133575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a6826147-b917-41c6-b8d6-ded706cf3af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:39.140511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b54116-a607-45fe-a8f8-a435e45a6139 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.141664 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1360/2736] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:39.145142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.146633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.146633 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.147049 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.147693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.149310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.153201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8084fa-c99f-457b-8d1e-c773790c767b None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.153495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8084fa-c99f-457b-8d1e-c773790c767b None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.153796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8084fa-c99f-457b-8d1e-c773790c767b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.166211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1377/2737] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:09:39.170446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.173803 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9f8084fa-c99f-457b-8d1e-c773790c767b None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:39.175009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9f8084fa-c99f-457b-8d1e-c773790c767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:39.180141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fcda3-8d2a-4721-9f8d-499c75744a69 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.183203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1361/2738] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:39.184899 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e19de-31e0-4b42-aa93-5198103fa058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.185621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e19de-31e0-4b42-aa93-5198103fa058 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.185925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e19de-31e0-4b42-aa93-5198103fa058 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.186430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e19de-31e0-4b42-aa93-5198103fa058 admin admin] PATH_INFO: `/v3/projects/4db4a0c8ef0a4db495d9f3144b4e3290/users/65472c21f17442d89ede36ebf3b5eb51/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.187275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8084fa-c99f-457b-8d1e-c773790c767b None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.196264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8084fa-c99f-457b-8d1e-c773790c767b None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.201414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e19de-31e0-4b42-aa93-5198103fa058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db4a0c8ef0a4db495d9f3144b4e3290, user_id=65472c21f17442d89ede36ebf3b5eb51, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.209914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.209914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.209914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.209914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.210333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:39.210444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e19de-31e0-4b42-aa93-5198103fa058 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.210539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.210539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.218164 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1362/2739] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:09:39.222403 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.233619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527b679-8f9b-4548-9269-57ac2258dc97 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.238874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.239560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.239841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.240116 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.240770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.242813 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.336552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1378/2740] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:39 2022] PUT /identity/v3/projects/4db4a0c8ef0a4db495d9f3144b4e3290/users/65472c21f17442d89ede36ebf3b5eb51/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:39.347221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113922-b3bd-4116-88cb-4378399e6f2f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.347509 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113922-b3bd-4116-88cb-4378399e6f2f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.347789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113922-b3bd-4116-88cb-4378399e6f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.368099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1363/2741] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:39.374225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.386472 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-ae113922-b3bd-4116-88cb-4378399e6f2f None None] MFA Rules not processed for user `65472c21f17442d89ede36ebf3b5eb51`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:39.387704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ae113922-b3bd-4116-88cb-4378399e6f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:39.423199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:39.447053 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de764ed-d495-4d9f-900e-8ff2820519cf admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.449866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.450604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.450889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.451210 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.451896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.454044 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.464941 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1379/2742] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:39.469282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae113922-b3bd-4116-88cb-4378399e6f2f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.481000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae113922-b3bd-4116-88cb-4378399e6f2f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.485672 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.486301 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.486584 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.486859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.487542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.490202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.591079 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1364/2743] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:39.592281 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.602983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3fcf3-dfe2-4624-9eb3-cbc048611b78 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.607732 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.608319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.608607 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.608903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.609561 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.611167 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.623695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1365/2744] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:39.625974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1380/2745] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:39.629861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.630815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.641517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6e274-ee3b-471d-babf-160e3f7b2294 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.698837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.699446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.699783 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.700059 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.700172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877c3ff-1521-49e1-95a8-1d70f9ec2cb2 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.701583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.703228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.708105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.708105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.708105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.708105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.709247 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.709963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.725811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1366/2746] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:39.728432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.738483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48cb15-d406-43ff-ba4c-1843959ec59c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.743216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41231a1-b8e8-4d27-82db-06f710a555ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.743858 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41231a1-b8e8-4d27-82db-06f710a555ef admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.744212 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41231a1-b8e8-4d27-82db-06f710a555ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.744418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41231a1-b8e8-4d27-82db-06f710a555ef admin admin] PATH_INFO: `/v3/projects/ee56dea141ed40ff909aaa07cc5ed8ab/users/27adc923785044bd838ee6311dd547d7/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.760359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41231a1-b8e8-4d27-82db-06f710a555ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee56dea141ed40ff909aaa07cc5ed8ab, user_id=27adc923785044bd838ee6311dd547d7, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.763074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41231a1-b8e8-4d27-82db-06f710a555ef admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.810994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1381/2747] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:39.814776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.824701 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b343b-8952-4988-9995-85ff397db296 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.829439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.830076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.830358 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.830630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.831273 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.832874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.834769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:39.844646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1382/2748] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:09:39.849564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:39.861378 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651be27-32bf-480e-8b2d-d5b83ed0c384 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:39.866713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:39.867433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.867713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.867991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] PATH_INFO: `/v3/projects/85816965caae4ffe9ed48b6ff97c2564/users/ad0f0cd339364fee86fd4d6fbf41f5b8/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.883239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85816965caae4ffe9ed48b6ff97c2564, user_id=ad0f0cd339364fee86fd4d6fbf41f5b8, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:39.885958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:39.946943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1367/2749] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:39 2022] PUT /identity/v3/projects/ee56dea141ed40ff909aaa07cc5ed8ab/users/27adc923785044bd838ee6311dd547d7/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:39.959126 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:39.959410 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:39.959760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:39.988404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] MFA Rules not processed for user `27adc923785044bd838ee6311dd547d7`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:40.008637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:40.056485 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1383/2750] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:39 2022] PUT /identity/v3/projects/85816965caae4ffe9ed48b6ff97c2564/users/ad0f0cd339364fee86fd4d6fbf41f5b8/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:40.057036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1368/2751] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:40.060480 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:40.070827 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:40.072376 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:40.131660 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a581517-5cfb-412c-ad17-dc7fb5ba5cd0 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:40.137749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:40.138681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:40.138839 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:40.139283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:40.139781 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:40.140656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:40.143081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:40.151026 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff6b35-d309-4726-8258-fe45309b8854 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:40.154875 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1384/2752] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:40.159157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:40.179000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b7fe7-e7da-4956-b5d2-71d8b899750d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:40.179000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5e74-dcd4-4402-830a-189088c2b89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:40.179000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5e74-dcd4-4402-830a-189088c2b89f admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:40.179000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5e74-dcd4-4402-830a-189088c2b89f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:40.179814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5e74-dcd4-4402-830a-189088c2b89f admin admin] PATH_INFO: `/v3/projects/85816965caae4ffe9ed48b6ff97c2564/users/ad0f0cd339364fee86fd4d6fbf41f5b8/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:40.179814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5e74-dcd4-4402-830a-189088c2b89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=85816965caae4ffe9ed48b6ff97c2564, user_id=ad0f0cd339364fee86fd4d6fbf41f5b8, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:40.182302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5e74-dcd4-4402-830a-189088c2b89f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:40.196313 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1385/2753] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:40 2022] PUT /identity/v3/projects/85816965caae4ffe9ed48b6ff97c2564/users/ad0f0cd339364fee86fd4d6fbf41f5b8/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:40.207271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247190bd-95cc-4edc-88e6-7b49e4143ab7 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:40.207552 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247190bd-95cc-4edc-88e6-7b49e4143ab7 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:40.209182 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247190bd-95cc-4edc-88e6-7b49e4143ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:40.215072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:40.215685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:40.219805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:40.219805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:40.239095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-247190bd-95cc-4edc-88e6-7b49e4143ab7 None None] MFA Rules not processed for user `ad0f0cd339364fee86fd4d6fbf41f5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:40.240356 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-247190bd-95cc-4edc-88e6-7b49e4143ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:40.285470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:40.287484 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:40.298109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1369/2754] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:40.320081 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1386/2755] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:40.327945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:40.329365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:40.341494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ff93-94db-4d37-8f05-76d7f116d178 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:40.399399 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:40.400003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:40.400276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:40.400543 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:40.410358 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:40.413874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:40.423848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1370/2756] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:41.800483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247190bd-95cc-4edc-88e6-7b49e4143ab7 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:41.810389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247190bd-95cc-4edc-88e6-7b49e4143ab7 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:41.816053 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad0f0cd339364fee86fd4d6fbf41f5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85816965caae4ffe9ed48b6ff97c2564', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:41.816683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:41.816973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:41.817256 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:41.818601 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:41.818916 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:41.820556 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:41.835414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1387/2757] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:41.852857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:41.852857 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb2246-86fd-4f62-97d4-366b1679a204 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:41.854310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad0f0cd339364fee86fd4d6fbf41f5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85816965caae4ffe9ed48b6ff97c2564', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:41.855040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:41.855359 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:41.855705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:41.856386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:41.858459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:42.014961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1371/2758] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:41 2022] POST /identity/v3/projects => generated 355 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:42.018044 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:42.029683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25d82-ca57-4c5f-b3b8-379faf738985 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:42.057920 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:42.090549 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad0f0cd339364fee86fd4d6fbf41f5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85816965caae4ffe9ed48b6ff97c2564', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:42.091158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:42.091439 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:42.091735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:42.092400 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:42.092703 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:42.101212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:42.106532 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1388/2759] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:42.144981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:42.156531 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3305be-ee9f-4ce5-97ab-3bcf961d25ee tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:42.162508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad0f0cd339364fee86fd4d6fbf41f5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85816965caae4ffe9ed48b6ff97c2564', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:42.163285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:42.163771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:42.164186 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] PATH_INFO: `/v3/projects/f7743837da6546e8996eeb7d0680b574` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:42.165475 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f7743837da6546e8996eeb7d0680b574)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:42.167494 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:42.176961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:42.424067 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7743837da6546e8996eeb7d0680b574', 'request_id': 'req-d8743532-c602-48a4-bcf4-20d98743bac3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:42.434049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7743837da6546e8996eeb7d0680b574', 'request_id': 'req-d8743532-c602-48a4-bcf4-20d98743bac3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:42.563854 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7743837da6546e8996eeb7d0680b574', 'request_id': 'req-d8743532-c602-48a4-bcf4-20d98743bac3'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:42.578304 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] The token cache is being invalidate because project f7743837da6546e8996eeb7d0680b574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:42.578600 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:42.579266 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8743532-c602-48a4-bcf4-20d98743bac3 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:42.580775 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1372/2760] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:42 2022] DELETE /identity/v3/projects/f7743837da6546e8996eeb7d0680b574 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:43.570076 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:43.571841 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:43.653964 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb554da-ef9e-483c-9577-1d884cbdec24 tempest-VolumeQuotaClassesTest-1338150475 tempest-VolumeQuotaClassesTest-1338150475-project-admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:43.709340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:43.710025 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:43.710401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:43.710647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] PATH_INFO: `/v3/users/65472c21f17442d89ede36ebf3b5eb51` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:43.711815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65472c21f17442d89ede36ebf3b5eb51)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:43.713723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:44.248312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65472c21f17442d89ede36ebf3b5eb51'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:44.257231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65472c21f17442d89ede36ebf3b5eb51'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:44.258261 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65472c21f17442d89ede36ebf3b5eb51'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:44.259596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65472c21f17442d89ede36ebf3b5eb51'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:44.413586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65472c21f17442d89ede36ebf3b5eb51'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:44.416527 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:44.428814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1389/2761] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:43 2022] DELETE /identity/v3/users/65472c21f17442d89ede36ebf3b5eb51 => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:44.436389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:44.436836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:44.437110 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:44.456201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:44.457416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:44.518608 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1373/2762] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:44.528751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:44.530283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:44.588634 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d1ee8-fe16-4a6f-9bd3-b044a5096001 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:44.592232 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:44.592834 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:44.593109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:44.593377 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:44.603288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:44.605038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:44.616740 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1390/2763] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:45.170078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:45.171495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:45.183703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1585f-b86d-4889-824b-2f4452191220 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:45.188422 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:45.189065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:45.189342 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:45.189618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] PATH_INFO: `/v3/projects/4db4a0c8ef0a4db495d9f3144b4e3290` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:45.190825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db4a0c8ef0a4db495d9f3144b4e3290)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:45.192767 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:45.451618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db4a0c8ef0a4db495d9f3144b4e3290', 'request_id': 'req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:45.457397 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db4a0c8ef0a4db495d9f3144b4e3290', 'request_id': 'req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:45.620611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db4a0c8ef0a4db495d9f3144b4e3290', 'request_id': 'req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:45.632898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] The token cache is being invalidate because project 4db4a0c8ef0a4db495d9f3144b4e3290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:45.633194 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:45.633937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:45.635429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1374/2764] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:45 2022] DELETE /identity/v3/projects/4db4a0c8ef0a4db495d9f3144b4e3290 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:45.639461 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:45.650112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142a6f6-43bf-40c0-a9ff-3717aeb90624 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:45.707519 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:45.708205 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:45.708670 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:45.709030 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] PATH_INFO: `/v3/users/ad0f0cd339364fee86fd4d6fbf41f5b8` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:45.710393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0f0cd339364fee86fd4d6fbf41f5b8)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:45.712504 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:46.007287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0f0cd339364fee86fd4d6fbf41f5b8'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:46.014406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0f0cd339364fee86fd4d6fbf41f5b8'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:46.015100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0f0cd339364fee86fd4d6fbf41f5b8'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:46.015710 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0f0cd339364fee86fd4d6fbf41f5b8'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:46.140816 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-25560a53-bb1e-4773-9cb7-6d2fc6e79aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0f0cd339364fee86fd4d6fbf41f5b8'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:46.159911 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1391/2765] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:45 2022] DELETE /identity/v3/users/ad0f0cd339364fee86fd4d6fbf41f5b8 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:46.678287 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:46.679814 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:46.693953 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d6250-dbf4-4c2d-8896-a80a9e68253f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:46.710507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:46.751277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:46.751983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:46.752248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:46.752523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] PATH_INFO: `/v3/projects/85816965caae4ffe9ed48b6ff97c2564` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:46.753679 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85816965caae4ffe9ed48b6ff97c2564)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:46.756588 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:46.913589 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85816965caae4ffe9ed48b6ff97c2564', 'request_id': 'req-0d8f439b-1258-48ed-a255-3540db801cc1'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:46.918942 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85816965caae4ffe9ed48b6ff97c2564', 'request_id': 'req-0d8f439b-1258-48ed-a255-3540db801cc1'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:46.998225 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85816965caae4ffe9ed48b6ff97c2564', 'request_id': 'req-0d8f439b-1258-48ed-a255-3540db801cc1'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:47.011030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] The token cache is being invalidate because project 85816965caae4ffe9ed48b6ff97c2564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:47.011316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:47.011946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-0d8f439b-1258-48ed-a255-3540db801cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:47.013471 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1375/2766] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:46 2022] DELETE /identity/v3/projects/85816965caae4ffe9ed48b6ff97c2564 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:47.018573 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722314c-9248-419b-bd63-018795d3bbc8 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.019091 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722314c-9248-419b-bd63-018795d3bbc8 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.019198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722314c-9248-419b-bd63-018795d3bbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.039499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-e722314c-9248-419b-bd63-018795d3bbc8 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:47.040706 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-e722314c-9248-419b-bd63-018795d3bbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:47.044852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:47.098973 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1392/2767] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:47.107004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24956-a7a9-4924-ae9c-add0b042a5af None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.107294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24956-a7a9-4924-ae9c-add0b042a5af None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.107570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24956-a7a9-4924-ae9c-add0b042a5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.127983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-faf24956-a7a9-4924-ae9c-add0b042a5af None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:47.134798 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-faf24956-a7a9-4924-ae9c-add0b042a5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:47.137631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1376/2768] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:47.141937 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722314c-9248-419b-bd63-018795d3bbc8 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:47.151567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722314c-9248-419b-bd63-018795d3bbc8 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:47.156198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:47.156800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.157075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.157347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.158035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:47.158338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:47.165202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:47.171943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1393/2769] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:09:47.176493 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf24956-a7a9-4924-ae9c-add0b042a5af None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:47.186804 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf24956-a7a9-4924-ae9c-add0b042a5af None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:47.191653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:47.192314 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.192633 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.192941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.193574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:47.195560 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:47.320062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1377/2770] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:47 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:47.324312 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:47.339935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d088de-414b-4b8e-9c02-7a2f501f850d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:47.394154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:47.394776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.395051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.395322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.396482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:47.398645 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:47.549229 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1394/2771] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:47 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:47.553448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:47.564434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c98b-6833-4906-9b9e-ebf9631bc323 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:47.569375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:47.570015 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.570298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.570565 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.571218 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:47.572842 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:47.584217 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1378/2772] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:47.588338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:47.609753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937aaa99-1342-498e-a545-e76cc024a099 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:47.609753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332d027-4173-4062-9542-4c87bb29734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:47.609753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d027-4173-4062-9542-4c87bb29734c admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.609753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d027-4173-4062-9542-4c87bb29734c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.610210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d027-4173-4062-9542-4c87bb29734c admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.610210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332d027-4173-4062-9542-4c87bb29734c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:47.610210 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332d027-4173-4062-9542-4c87bb29734c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:47.618539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1395/2773] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:47.622859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:47.633524 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bcf6d-2c91-4994-9178-16ece2ae612c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:47.638228 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:47.638881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.639162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.639437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] PATH_INFO: `/v3/projects/212f31f75bc848eab426682acc1603b1/users/8b71d1411be743cd970935aa5a431906/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.655707 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212f31f75bc848eab426682acc1603b1, user_id=8b71d1411be743cd970935aa5a431906, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:47.659004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:47.765265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1379/2774] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:47 2022] PUT /identity/v3/projects/212f31f75bc848eab426682acc1603b1/users/8b71d1411be743cd970935aa5a431906/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:47.783391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ad2bf-b7ed-49d4-bde3-6a6d03962568 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.783761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ad2bf-b7ed-49d4-bde3-6a6d03962568 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.784127 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ad2bf-b7ed-49d4-bde3-6a6d03962568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.812981 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b59ad2bf-b7ed-49d4-bde3-6a6d03962568 None None] MFA Rules not processed for user `8b71d1411be743cd970935aa5a431906`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:47.814307 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b59ad2bf-b7ed-49d4-bde3-6a6d03962568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:47.883000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1396/2775] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:47.887309 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:47.940877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:47.955448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a137c1-fb58-4b3c-9543-df0dc5b194c4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:47.960330 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:47.960940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:47.961220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:47.961547 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:47.962220 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:47.964128 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:48.103526 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1380/2776] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:47 2022] POST /identity/v3/projects => generated 387 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:48.107605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ad2bf-b7ed-49d4-bde3-6a6d03962568 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:48.114935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:48.119338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ad2bf-b7ed-49d4-bde3-6a6d03962568 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:48.184165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:48.184791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:48.185096 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:48.185371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:48.186051 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:48.187963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:48.387664 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1397/2777] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:48 2022] POST /identity/v3/users => generated 300 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:48.391495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:48.402983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1889-36f9-467b-bc37-719348f786bb admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:48.408045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:48.408711 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:48.408975 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:48.409249 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:48.410061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:48.412255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:48.423175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1381/2778] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:09:48.427394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:48.437450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ed65-fb5f-4714-a5da-545c4c33d772 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:48.449614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:48.449614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:48.449614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:48.449614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] PATH_INFO: `/v3/projects/abe8d381646949f99fd89598ef98f311/users/bfc012c0a25b4289a94cc8c9adbda7e2/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:48.459335 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe8d381646949f99fd89598ef98f311, user_id=bfc012c0a25b4289a94cc8c9adbda7e2, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:48.462165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:48.581972 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1398/2779] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:48 2022] PUT /identity/v3/projects/abe8d381646949f99fd89598ef98f311/users/bfc012c0a25b4289a94cc8c9adbda7e2/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:48.588310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:48.599119 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7d0c0-4aac-4331-83da-acc7f366823e admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:48.666846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf776b1-a6ca-431c-93bb-f08138404d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:48.667660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf776b1-a6ca-431c-93bb-f08138404d2b admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:48.667943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf776b1-a6ca-431c-93bb-f08138404d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:48.668215 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf776b1-a6ca-431c-93bb-f08138404d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:48.668866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf776b1-a6ca-431c-93bb-f08138404d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:48.670499 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf776b1-a6ca-431c-93bb-f08138404d2b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:48.684624 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1382/2780] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:48.686055 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:48.696836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f339e-51da-4fde-a074-156b6c1ff25c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:48.701428 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:48.702220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:48.702498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:48.702769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] PATH_INFO: `/v3/projects/abe8d381646949f99fd89598ef98f311/users/bfc012c0a25b4289a94cc8c9adbda7e2/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:48.704789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe8d381646949f99fd89598ef98f311, user_id=bfc012c0a25b4289a94cc8c9adbda7e2, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:48.707503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:48.718063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1399/2781] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:48 2022] PUT /identity/v3/projects/abe8d381646949f99fd89598ef98f311/users/bfc012c0a25b4289a94cc8c9adbda7e2/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:48.730179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b924ac-4d6a-4db6-a2b6-988c3f31baf6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:48.730473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b924ac-4d6a-4db6-a2b6-988c3f31baf6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:48.730771 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b924ac-4d6a-4db6-a2b6-988c3f31baf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:48.765627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-12b924ac-4d6a-4db6-a2b6-988c3f31baf6 None None] MFA Rules not processed for user `bfc012c0a25b4289a94cc8c9adbda7e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:48.766891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-12b924ac-4d6a-4db6-a2b6-988c3f31baf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:48.843195 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1383/2782] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:48.853947 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:48.855938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:48.918213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce21bd0-903f-4625-8dee-d1f02cb1a132 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:48.921908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b14a6-7cad-4ce0-bd21-b92212cf93ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:48.922516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b14a6-7cad-4ce0-bd21-b92212cf93ca service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:48.922789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b14a6-7cad-4ce0-bd21-b92212cf93ca service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:48.923066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b14a6-7cad-4ce0-bd21-b92212cf93ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:48.933098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b14a6-7cad-4ce0-bd21-b92212cf93ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:48.934571 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b14a6-7cad-4ce0-bd21-b92212cf93ca service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:48.945889 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1400/2783] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:50.717041 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b924ac-4d6a-4db6-a2b6-988c3f31baf6 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:50.766640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:50.785027 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b924ac-4d6a-4db6-a2b6-988c3f31baf6 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:50.790492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:50.791124 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:50.791405 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:50.791678 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] PATH_INFO: `/v3/users/8b71d1411be743cd970935aa5a431906` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:50.792846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b71d1411be743cd970935aa5a431906)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:50.794801 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:51.120690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b71d1411be743cd970935aa5a431906'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:51.127529 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b71d1411be743cd970935aa5a431906'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:51.216256 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b71d1411be743cd970935aa5a431906'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:51.216908 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b71d1411be743cd970935aa5a431906'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:51.237424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b71d1411be743cd970935aa5a431906'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:51.244267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1384/2784] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:50 2022] DELETE /identity/v3/users/8b71d1411be743cd970935aa5a431906 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:51.249189 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:51.250202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:51.250531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:51.273635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:51.274921 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:51.326362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:51.335763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1401/2785] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:51.345838 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:51.347317 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:51.407327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0e1a4-971e-4b28-8f0f-fb38d9cf1f0f admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:51.411592 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:51.412239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:51.412520 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:51.412791 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:51.421993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:51.423428 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:51.433101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1385/2786] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:51.999751 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:52.001281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:52.015935 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7079d-8440-445a-b3ea-dbf413cb9a6a None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:52.020919 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:52.021567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:52.022424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:52.022717 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] PATH_INFO: `/v3/projects/212f31f75bc848eab426682acc1603b1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:52.024062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=212f31f75bc848eab426682acc1603b1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:52.026220 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:52.218585 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212f31f75bc848eab426682acc1603b1', 'request_id': 'req-a535bd0c-7dd4-4ebd-9978-15d5dc544142'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:52.226464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212f31f75bc848eab426682acc1603b1', 'request_id': 'req-a535bd0c-7dd4-4ebd-9978-15d5dc544142'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:52.232733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212f31f75bc848eab426682acc1603b1', 'request_id': 'req-a535bd0c-7dd4-4ebd-9978-15d5dc544142'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:52.399032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] The token cache is being invalidate because project 212f31f75bc848eab426682acc1603b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:52.399669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:52.399929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:52.401544 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1402/2787] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:51 2022] DELETE /identity/v3/projects/212f31f75bc848eab426682acc1603b1 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:52.405503 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:52.415293 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3d7b1-5381-4285-887e-06208024b5ef service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:52.474495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:52.475260 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:52.475700 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:52.475817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] PATH_INFO: `/v3/users/bfc012c0a25b4289a94cc8c9adbda7e2` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:52.476990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc012c0a25b4289a94cc8c9adbda7e2)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:52.478993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:52.484381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:52.806509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc012c0a25b4289a94cc8c9adbda7e2'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:52.813316 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc012c0a25b4289a94cc8c9adbda7e2'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:53.021675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc012c0a25b4289a94cc8c9adbda7e2'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:53.022684 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc012c0a25b4289a94cc8c9adbda7e2'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:53.035563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc012c0a25b4289a94cc8c9adbda7e2'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:53.042065 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1386/2788] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:52 2022] DELETE /identity/v3/users/bfc012c0a25b4289a94cc8c9adbda7e2 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:53.932145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:53.935665 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:53.948994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535bd0c-7dd4-4ebd-9978-15d5dc544142 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.009101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.012889 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:54.029260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.029940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.030219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.030490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] PATH_INFO: `/v3/projects/abe8d381646949f99fd89598ef98f311` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.031823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe8d381646949f99fd89598ef98f311)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.033838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.091258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a058a-d8b5-4466-98fe-4d5df3a655fd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.095966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.096578 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.096863 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.097138 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] PATH_INFO: `/v3/users/27adc923785044bd838ee6311dd547d7` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.098354 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27adc923785044bd838ee6311dd547d7)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.100250 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.208057 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe8d381646949f99fd89598ef98f311', 'request_id': 'req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.214466 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe8d381646949f99fd89598ef98f311', 'request_id': 'req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.220159 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe8d381646949f99fd89598ef98f311', 'request_id': 'req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.336349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] The token cache is being invalidate because project abe8d381646949f99fd89598ef98f311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:54.336349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.336349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-84e446ea-3ab4-4140-84ea-9ea36fe9c3a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.336349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1403/2789] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:53 2022] DELETE /identity/v3/projects/abe8d381646949f99fd89598ef98f311 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:54.336349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920a0b-580b-436d-a5e3-9445a3b0af22 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.336823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920a0b-580b-436d-a5e3-9445a3b0af22 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.336823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920a0b-580b-436d-a5e3-9445a3b0af22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.356191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-a4920a0b-580b-436d-a5e3-9445a3b0af22 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:54.357401 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a4920a0b-580b-436d-a5e3-9445a3b0af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:54.367119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:54.418254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27adc923785044bd838ee6311dd547d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.424311 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1404/2790] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:54.425472 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27adc923785044bd838ee6311dd547d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.431110 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da320705-4ac2-451e-9894-4ccebe22914f None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.431405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da320705-4ac2-451e-9894-4ccebe22914f None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.431684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da320705-4ac2-451e-9894-4ccebe22914f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.451847 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-da320705-4ac2-451e-9894-4ccebe22914f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:54.453023 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-da320705-4ac2-451e-9894-4ccebe22914f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:54.461395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1405/2791] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:54.465508 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da320705-4ac2-451e-9894-4ccebe22914f None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.475158 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da320705-4ac2-451e-9894-4ccebe22914f None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.480047 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.480632 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.480910 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.481187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.481870 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:54.482198 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.483849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.496219 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1406/2792] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:09:54.500338 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.510821 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f0780-362c-492c-be2a-b9d71a60bcbb admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.515460 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.516087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.516371 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.516644 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.517345 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.517591 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27adc923785044bd838ee6311dd547d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.518251 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27adc923785044bd838ee6311dd547d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.519287 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.531937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d9587774-56a5-428d-acb2-947dc8ca4cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27adc923785044bd838ee6311dd547d7'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:54.537916 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1387/2793] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:09:54 2022] DELETE /identity/v3/users/27adc923785044bd838ee6311dd547d7 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:54.542302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.542594 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.542866 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.549881 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:54.563772 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:54.567705 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:54.623082 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1388/2794] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:09:54.633627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.639507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:54.639651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1407/2795] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:54 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:54.640353 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.642555 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:54.695548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbcde3-7915-4d49-9bc3-56141113e455 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.711594 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25db37b-a5b0-4f7a-a2b4-4679793f3cbf admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.716201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.716851 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.717516 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.718145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.719113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.721355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.744654 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.745259 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.745571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.745875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.755800 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.757216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.766630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1389/2796] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:09:54.827617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1408/2797] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:54 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:09:54.832313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.847278 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159d4ff-cdd7-4701-92ed-ab64688d6f02 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.852062 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.852667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.852945 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.853329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.854528 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.856150 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.867268 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1390/2798] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:54.871241 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.882129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac617e-1aa9-4774-8d2e-2df7ff9c4121 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.886683 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947b467-1643-40aa-a2bd-6e687c387500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.887269 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947b467-1643-40aa-a2bd-6e687c387500 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.887546 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947b467-1643-40aa-a2bd-6e687c387500 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.887817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947b467-1643-40aa-a2bd-6e687c387500 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.888450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947b467-1643-40aa-a2bd-6e687c387500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.890077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947b467-1643-40aa-a2bd-6e687c387500 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:54.900348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1409/2799] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:54.904444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:54.915020 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806693-dfda-4575-9bdf-7e3858675dd1 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:54.919550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:54.926169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:54.926169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:54.926169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] PATH_INFO: `/v3/projects/b51460a55a6a4647ae6c4f7590ca053a/users/c9f2df5ac7a04c0d9d6db94df0c068db/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:54.935011 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51460a55a6a4647ae6c4f7590ca053a, user_id=c9f2df5ac7a04c0d9d6db94df0c068db, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:54.937724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:55.158859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1391/2800] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:54 2022] PUT /identity/v3/projects/b51460a55a6a4647ae6c4f7590ca053a/users/c9f2df5ac7a04c0d9d6db94df0c068db/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:55.167828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.168443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.168611 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.198126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] MFA Rules not processed for user `c9f2df5ac7a04c0d9d6db94df0c068db`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:55.199505 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:55.239627 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:55.263856 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1410/2801] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:55.275761 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:55.277532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:55.307947 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:55.335957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546a5ff-12d1-4752-8803-492c08e92125 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:55.339545 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4462c-76fe-4012-bb7b-04fec5726809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:55.340132 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4462c-76fe-4012-bb7b-04fec5726809 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.340404 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4462c-76fe-4012-bb7b-04fec5726809 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.340667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4462c-76fe-4012-bb7b-04fec5726809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.348981 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4462c-76fe-4012-bb7b-04fec5726809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:55.350462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4462c-76fe-4012-bb7b-04fec5726809 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:55.355869 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:55.357734 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:55.360092 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1392/2802] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:55.439482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a136-ee83-47ea-9bce-5fe6c8fde073 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:55.444864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:55.445582 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.445985 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.446325 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] PATH_INFO: `/v3/projects/ee56dea141ed40ff909aaa07cc5ed8ab` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.448066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee56dea141ed40ff909aaa07cc5ed8ab)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:55.449498 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:55.546867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee56dea141ed40ff909aaa07cc5ed8ab', 'request_id': 'req-33387def-759d-441e-998f-40b185e72ffa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:55.553599 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee56dea141ed40ff909aaa07cc5ed8ab', 'request_id': 'req-33387def-759d-441e-998f-40b185e72ffa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:55.559099 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee56dea141ed40ff909aaa07cc5ed8ab', 'request_id': 'req-33387def-759d-441e-998f-40b185e72ffa'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:55.717968 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] The token cache is being invalidate because project ee56dea141ed40ff909aaa07cc5ed8ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:09:55.718176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:55.718817 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-33387def-759d-441e-998f-40b185e72ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:09:55.720354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1411/2803] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:09:55 2022] DELETE /identity/v3/projects/ee56dea141ed40ff909aaa07cc5ed8ab => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:09:55.725925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208dfa8-0fb7-4104-990a-6956bec88138 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.726232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208dfa8-0fb7-4104-990a-6956bec88138 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.726504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208dfa8-0fb7-4104-990a-6956bec88138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.746163 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-e208dfa8-0fb7-4104-990a-6956bec88138 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:55.747379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-e208dfa8-0fb7-4104-990a-6956bec88138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:55.807659 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1393/2804] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:55.815469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4672a-d9f9-4935-a881-ca65644658f6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.815756 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4672a-d9f9-4935-a881-ca65644658f6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.816032 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4672a-d9f9-4935-a881-ca65644658f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.837322 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-27b4672a-d9f9-4935-a881-ca65644658f6 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:55.838577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-27b4672a-d9f9-4935-a881-ca65644658f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:55.846904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1412/2805] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:55.851998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208dfa8-0fb7-4104-990a-6956bec88138 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:55.861909 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208dfa8-0fb7-4104-990a-6956bec88138 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:55.867176 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:55.867780 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.868058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.868328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.868974 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:09:55.869276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:55.870968 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:55.883781 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1394/2806] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:09:55.888538 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4672a-d9f9-4935-a881-ca65644658f6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:55.898867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4672a-d9f9-4935-a881-ca65644658f6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:55.903758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:55.904389 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:55.904681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:55.904945 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:55.905617 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:55.907562 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:56.026447 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1413/2807] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:09:55 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:56.027880 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:56.038698 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08cb89-b2ed-4736-906e-5fe7c6d46283 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:56.095756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:56.096367 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:56.096650 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:56.096925 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:56.097569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:56.100107 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:56.232696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1395/2808] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:09:56 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:09:56.237115 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:56.248184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea6ec5-20aa-4e39-a66a-4aa59e2eefc1 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:56.252868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:56.253471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:56.253748 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:56.254059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:56.255499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:56.257088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:56.269007 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1414/2809] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:09:56.273750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:56.285498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0896523-4dee-4946-9ff9-9880a9bc0881 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:56.291070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82334a-08dc-4272-8bad-104d6b5a8d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:56.291896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82334a-08dc-4272-8bad-104d6b5a8d40 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:56.292270 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82334a-08dc-4272-8bad-104d6b5a8d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:56.292606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82334a-08dc-4272-8bad-104d6b5a8d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:56.293450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82334a-08dc-4272-8bad-104d6b5a8d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:56.295402 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82334a-08dc-4272-8bad-104d6b5a8d40 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:56.308161 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1396/2810] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:09:56.312828 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:56.339283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e198be-3dc2-476b-af1f-0fb862d493a5 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:56.339283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:56.339283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:56.339283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:56.340288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] PATH_INFO: `/v3/projects/442a8080466045d1ac6d252642b56208/users/1d333e453a8d4df8a1f58d6ffe30fce7/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:56.344163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=442a8080466045d1ac6d252642b56208, user_id=1d333e453a8d4df8a1f58d6ffe30fce7, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:56.346848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:56.563877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1415/2811] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:09:56 2022] PUT /identity/v3/projects/442a8080466045d1ac6d252642b56208/users/1d333e453a8d4df8a1f58d6ffe30fce7/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:09:56.575768 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433a7ea-9a20-4d92-83b3-44d466afbb16 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:56.576051 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433a7ea-9a20-4d92-83b3-44d466afbb16 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:56.576324 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433a7ea-9a20-4d92-83b3-44d466afbb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:56.603640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-b433a7ea-9a20-4d92-83b3-44d466afbb16 None None] MFA Rules not processed for user `1d333e453a8d4df8a1f58d6ffe30fce7`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:56.604873 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b433a7ea-9a20-4d92-83b3-44d466afbb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:56.628694 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:56.669179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1397/2812] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:09:56.679899 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:09:56.681681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:56.691392 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:09:56.740833 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9a74a-1946-4d69-9000-df97af91b39c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:09:56.744470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:09:56.745095 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:56.745368 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:56.745635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:56.753815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:09:56.755966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:09:56.764648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1416/2813] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:09:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:09:58.549134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64192fb1-8fc5-4810-9ef8-eb5e0beb1014 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:09:58.549651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64192fb1-8fc5-4810-9ef8-eb5e0beb1014 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:09:58.549808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64192fb1-8fc5-4810-9ef8-eb5e0beb1014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:09:58.571040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-64192fb1-8fc5-4810-9ef8-eb5e0beb1014 None None] MFA Rules not processed for user `1d333e453a8d4df8a1f58d6ffe30fce7`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:09:58.572174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-64192fb1-8fc5-4810-9ef8-eb5e0beb1014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:09:58.579300 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1398/2814] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:09:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:07.346823 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=23 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:08.563864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=19 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:25.529354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:25.530811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:25.595197 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a95b2-c75b-480a-a205-d5ffe22c5cd7 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:25.600286 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:25.600904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:25.601176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:25.601446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] PATH_INFO: `/v3/users/c9f2df5ac7a04c0d9d6db94df0c068db` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:25.602614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f2df5ac7a04c0d9d6db94df0c068db)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:25.604518 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:25.800731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f2df5ac7a04c0d9d6db94df0c068db'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:25.807605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f2df5ac7a04c0d9d6db94df0c068db'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:25.808363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f2df5ac7a04c0d9d6db94df0c068db'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:25.808965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f2df5ac7a04c0d9d6db94df0c068db'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:25.883157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f2df5ac7a04c0d9d6db94df0c068db'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:25.902193 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1417/2815] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:10:25 2022] DELETE /identity/v3/users/c9f2df5ac7a04c0d9d6db94df0c068db => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:25.906658 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:25.907185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:25.907185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:25.926697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:25.927895 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:25.986197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1399/2816] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:10:25.995877 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:25.997291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:26.055965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224af62d-6b37-4272-b9ac-ea9102dddd8a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:26.059712 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:26.060321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:26.060596 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:26.060865 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:26.071011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:26.072450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:26.085446 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1418/2817] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:26.501495 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:26.503074 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:26.516301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d966b1d-89a2-4d46-ad45-020c623b66f4 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:26.521134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:26.521831 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:26.522185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:26.522457 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] PATH_INFO: `/v3/projects/b51460a55a6a4647ae6c4f7590ca053a` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:26.523708 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b51460a55a6a4647ae6c4f7590ca053a)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:26.525946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:26.626927 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51460a55a6a4647ae6c4f7590ca053a', 'request_id': 'req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:26.633302 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51460a55a6a4647ae6c4f7590ca053a', 'request_id': 'req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:26.711682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51460a55a6a4647ae6c4f7590ca053a', 'request_id': 'req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:26.724328 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] The token cache is being invalidate because project b51460a55a6a4647ae6c4f7590ca053a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:10:26.724614 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:26.725267 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-07c2db7d-16f6-4cf9-bf4c-bb7ef34c0afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:26.726788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1400/2818] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:10:26 2022] DELETE /identity/v3/projects/b51460a55a6a4647ae6c4f7590ca053a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:26.731317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:26.732793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:26.733223 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b57c-cc98-4915-a43a-dcd781782991 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:26.733512 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b57c-cc98-4915-a43a-dcd781782991 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:26.733812 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b57c-cc98-4915-a43a-dcd781782991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:26.755177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-23a7b57c-cc98-4915-a43a-dcd781782991 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:26.756378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-23a7b57c-cc98-4915-a43a-dcd781782991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:26.766713 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:26.802203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8d40-b240-4e4b-8e8d-fde7e953c5dc service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:26.825298 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1401/2819] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:26.833057 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef42a9-01ee-4805-977f-9424c2b12511 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:26.833344 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef42a9-01ee-4805-977f-9424c2b12511 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:26.833619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef42a9-01ee-4805-977f-9424c2b12511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:26.852484 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:26.853109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:26.853391 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:26.853895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:26.855341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-18ef42a9-01ee-4805-977f-9424c2b12511 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:26.855656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:26.856629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-18ef42a9-01ee-4805-977f-9424c2b12511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:26.866335 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1402/2820] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:10:26.870788 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef42a9-01ee-4805-977f-9424c2b12511 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:26.880758 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef42a9-01ee-4805-977f-9424c2b12511 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:26.887551 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:26.888174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:26.888443 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:26.888795 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:26.889479 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:10:26.889896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:26.891507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:26.904085 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1403/2821] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 14:10:26.910864 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:26.917821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:26.921733 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502dc4-5d12-4a45-aa5d-80c61a805bd7 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:26.922482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:26.924000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:26.927486 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:26.928087 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:26.928535 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:26.928673 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:26.929318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:26.931273 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:26.937914 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1419/2822] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:10:27.051069 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1404/2823] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:10:26 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:10:27.055120 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:27.065693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacf5b-96e6-4fa5-a58f-810d3c0bc5f0 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:27.128708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:27.129456 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:27.130038 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:27.130321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:27.131031 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:27.133542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:27.291746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1420/2824] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:10:27 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:10:27.300322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:27.320327 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91c2de-3555-43f3-9693-63adf50ab3f9 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:27.327049 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:27.327819 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:27.328105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:27.328381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:27.329094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:27.332571 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:27.347936 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1405/2825] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:10:27.350614 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:27.362861 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faca27-96a9-43e0-8784-90b4d9f2287d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:27.369088 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5685aec-ccf3-4c8a-a2d3-c6e43bde24ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:27.369708 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5685aec-ccf3-4c8a-a2d3-c6e43bde24ee admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:27.369879 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5685aec-ccf3-4c8a-a2d3-c6e43bde24ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:27.370146 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5685aec-ccf3-4c8a-a2d3-c6e43bde24ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:27.370891 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5685aec-ccf3-4c8a-a2d3-c6e43bde24ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:27.372422 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5685aec-ccf3-4c8a-a2d3-c6e43bde24ee admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:27.389372 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1421/2826] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:10:27.392663 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:27.404882 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e098d-e897-4385-9796-82123088d2c6 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:27.412454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:27.413148 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:27.413431 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:27.413734 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] PATH_INFO: `/v3/projects/bd1cae0cf4124ec392459a4ce7c234bf/users/392ba34402bb49278ee12c6bf89715d6/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:27.430590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1cae0cf4124ec392459a4ce7c234bf, user_id=392ba34402bb49278ee12c6bf89715d6, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:27.433294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:27.646487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1406/2827] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:10:27 2022] PUT /identity/v3/projects/bd1cae0cf4124ec392459a4ce7c234bf/users/392ba34402bb49278ee12c6bf89715d6/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:27.656946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295f6c-0924-4b53-a112-96f614dd2bb6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:27.657511 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295f6c-0924-4b53-a112-96f614dd2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:27.658087 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295f6c-0924-4b53-a112-96f614dd2bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:27.692223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:27.707651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-3f295f6c-0924-4b53-a112-96f614dd2bb6 None None] MFA Rules not processed for user `392ba34402bb49278ee12c6bf89715d6`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:27.707651 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3f295f6c-0924-4b53-a112-96f614dd2bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:27.801885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1422/2828] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:27.812784 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:27.895640 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906afad-3c14-45b2-9723-d28119885331 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:27.900434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:27.901232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:27.901334 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:27.901525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:27.902290 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:27.904264 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:28.068045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1407/2829] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:10:27 2022] POST /identity/v3/projects => generated 383 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:10:28.072419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f295f6c-0924-4b53-a112-96f614dd2bb6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:28.084123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f295f6c-0924-4b53-a112-96f614dd2bb6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:28.142536 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:28.143145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.143421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.143690 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:28.144958 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:28.146955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:28.266635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1423/2830] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:10:28 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:10:28.270995 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:28.282295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c6e1-be98-412a-85bd-3ee0cc32aca4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:28.287030 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:28.287632 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.287970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.288242 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:28.288890 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:28.290662 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:28.301590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1408/2831] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 14:10:28.305648 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:28.316570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb83563-db76-438a-9218-08feabedfd1d admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:28.321362 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8037291-0105-4516-9075-1cc602a9766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:28.322075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037291-0105-4516-9075-1cc602a9766b admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.322386 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037291-0105-4516-9075-1cc602a9766b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.322663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037291-0105-4516-9075-1cc602a9766b admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:28.323347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8037291-0105-4516-9075-1cc602a9766b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:28.324992 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8037291-0105-4516-9075-1cc602a9766b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:28.336026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1424/2832] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:10:28.340148 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:28.350329 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bbf72-a1f0-42ba-aa34-d98fb69d67ae admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:28.355003 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:28.355682 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.355964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.356285 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] PATH_INFO: `/v3/projects/b9f0d3accae145d18e928811c77c29c2/users/6f938e411eaf43e483532c42d8ccb4fd/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:28.372394 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f0d3accae145d18e928811c77c29c2, user_id=6f938e411eaf43e483532c42d8ccb4fd, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:28.375006 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:28.467040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1409/2833] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:10:28 2022] PUT /identity/v3/projects/b9f0d3accae145d18e928811c77c29c2/users/6f938e411eaf43e483532c42d8ccb4fd/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:28.479098 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2a934-77b0-49df-ad64-181d4c1d9531 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.479388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2a934-77b0-49df-ad64-181d4c1d9531 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.479667 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2a934-77b0-49df-ad64-181d4c1d9531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:28.509721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-1de2a934-77b0-49df-ad64-181d4c1d9531 None None] MFA Rules not processed for user `6f938e411eaf43e483532c42d8ccb4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:28.510987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1de2a934-77b0-49df-ad64-181d4c1d9531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:28.577267 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1425/2834] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:28.581793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:28.634709 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:28.648341 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8028368-25f1-416c-9a0d-2b4d02faa0da admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:28.655178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:28.655178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.655178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.655178 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:28.655590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:28.657033 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:28.793871 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1410/2835] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:10:28 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:10:28.798173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2a934-77b0-49df-ad64-181d4c1d9531 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:28.809827 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2a934-77b0-49df-ad64-181d4c1d9531 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:28.814986 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:28.871499 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:28.872464 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.872744 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.873011 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:28.873658 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:28.875657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:28.978183 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1426/2836] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:10:28 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:10:28.982245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:28.993784 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241fead-f1fa-4437-a11d-2d7ee5704123 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:28.998554 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:28.999177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:28.999454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:28.999722 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:29.000375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:29.001996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:29.013190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1411/2837] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:10:29.017711 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:29.030788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898371a-8594-4eea-a65d-45a874b0ce95 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:29.036129 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:29.036866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:29.037113 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:29.037383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] PATH_INFO: `/v3/projects/67811bc54dcc48cb95a94dabcdd2fc07/users/88d4ad64803946d5ac1fcf05813c76fb/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:29.054240 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] RBAC: Authorizing `identity:create_grant(project_id=67811bc54dcc48cb95a94dabcdd2fc07, user_id=88d4ad64803946d5ac1fcf05813c76fb, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:29.057016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:29.193785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1427/2838] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:10:29 2022] PUT /identity/v3/projects/67811bc54dcc48cb95a94dabcdd2fc07/users/88d4ad64803946d5ac1fcf05813c76fb/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:29.198028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:29.208943 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163212b-912b-498e-8464-18ba72875a71 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:29.272562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd099637-2589-4ce5-a24e-4759a14ac7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:29.273196 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099637-2589-4ce5-a24e-4759a14ac7bf admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:29.273502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099637-2589-4ce5-a24e-4759a14ac7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:29.273805 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099637-2589-4ce5-a24e-4759a14ac7bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:29.274436 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd099637-2589-4ce5-a24e-4759a14ac7bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:29.276078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd099637-2589-4ce5-a24e-4759a14ac7bf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:29.288445 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1412/2839] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:10:29.292691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:29.306567 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5224e-eed6-4604-a909-fd9d0e8e901a admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:29.311989 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:29.312662 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:29.312942 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:29.313226 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] PATH_INFO: `/v3/projects/67811bc54dcc48cb95a94dabcdd2fc07/users/88d4ad64803946d5ac1fcf05813c76fb/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:29.315388 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67811bc54dcc48cb95a94dabcdd2fc07, user_id=88d4ad64803946d5ac1fcf05813c76fb, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:29.318026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:29.332363 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1428/2840] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:10:29 2022] PUT /identity/v3/projects/67811bc54dcc48cb95a94dabcdd2fc07/users/88d4ad64803946d5ac1fcf05813c76fb/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:10:29.343394 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:29.343704 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:29.343978 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:29.373901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] MFA Rules not processed for user `88d4ad64803946d5ac1fcf05813c76fb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:29.375131 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:29.448934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1413/2841] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:29.461117 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:29.463339 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:29.523195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9e01-4c3d-43c0-b155-cac43da5aa60 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:29.526836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:29.527471 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:29.527749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:29.528054 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:29.586154 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:29.587600 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:29.596776 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1429/2842] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:31.130690 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:31.132199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:31.182891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:31.195810 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d124169-6c9b-43df-a8c3-30b69ca29069 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:31.199430 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ca882-b0ab-42b4-aa46-0091466e820b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:31.200031 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ca882-b0ab-42b4-aa46-0091466e820b service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:31.200307 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ca882-b0ab-42b4-aa46-0091466e820b service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:31.200581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ca882-b0ab-42b4-aa46-0091466e820b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:31.202628 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8d8ca882-b0ab-42b4-aa46-0091466e820b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:31.292478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ca882-b0ab-42b4-aa46-0091466e820b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:31.293964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ca882-b0ab-42b4-aa46-0091466e820b service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:31.315356 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1414/2843] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:31.791520 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:31.800959 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458376e-06dc-4fe9-b56b-0e592482f7c5 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:31.805033 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:31.805655 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:31.805967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:31.806276 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:31.820753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:31.888306 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:31.889788 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:31.905840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1430/2844] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:32.934312 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca425d4-a3e9-47cd-b289-0fb5b6e02d9d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:32.934784 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca425d4-a3e9-47cd-b289-0fb5b6e02d9d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:32.934878 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca425d4-a3e9-47cd-b289-0fb5b6e02d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:32.965292 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-7ca425d4-a3e9-47cd-b289-0fb5b6e02d9d None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:32.972647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7ca425d4-a3e9-47cd-b289-0fb5b6e02d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:33.025125 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1415/2845] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:10:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:33.373344 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:33.374778 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:33.447840 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb416f4b-ef44-4d3c-acf8-e39dc4ae6049 service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:33.452723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7241cf-5251-4b93-9476-b75dd9cf1c19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:33.453334 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7241cf-5251-4b93-9476-b75dd9cf1c19 service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:33.453605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7241cf-5251-4b93-9476-b75dd9cf1c19 service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:33.453907 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7241cf-5251-4b93-9476-b75dd9cf1c19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:33.463163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7241cf-5251-4b93-9476-b75dd9cf1c19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:33.464779 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7241cf-5251-4b93-9476-b75dd9cf1c19 service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:33.474970 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1431/2846] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:10:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:10:33.980948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca425d4-a3e9-47cd-b289-0fb5b6e02d9d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:33.980948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca425d4-a3e9-47cd-b289-0fb5b6e02d9d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:33.980948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:33.980948 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:33.981735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:33.981735 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:33.983230 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:33.997173 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:33.998627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:34.010562 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1416/2847] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:10:34.671540 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ca7d8-24dd-4cf4-bffd-c6abfb2c1bd6 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:34.672035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ca7d8-24dd-4cf4-bffd-c6abfb2c1bd6 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:34.672142 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ca7d8-24dd-4cf4-bffd-c6abfb2c1bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:34.692017 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4f4ca7d8-24dd-4cf4-bffd-c6abfb2c1bd6 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:34.693245 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4f4ca7d8-24dd-4cf4-bffd-c6abfb2c1bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:34.700913 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1432/2848] 10.222.0.61 () {58 vars in 967 bytes} [Thu Sep 1 14:10:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:34.713627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:34.726598 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ca7d8-24dd-4cf4-bffd-c6abfb2c1bd6 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:34.729094 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d0a0f-6d3b-4e05-b7e3-cbc94a399624 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:34.734276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147c6b0-aa28-4033-aa13-e48a90f371d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:34.734907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147c6b0-aa28-4033-aa13-e48a90f371d0 service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:34.735189 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147c6b0-aa28-4033-aa13-e48a90f371d0 service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:34.735466 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147c6b0-aa28-4033-aa13-e48a90f371d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:34.735700 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ca7d8-24dd-4cf4-bffd-c6abfb2c1bd6 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:34.741156 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:34.741867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:34.742231 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:34.742586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:34.750133 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147c6b0-aa28-4033-aa13-e48a90f371d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:34.751998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147c6b0-aa28-4033-aa13-e48a90f371d0 service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:34.752160 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:34.753811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:34.760584 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1417/2849] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:10:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:10:34.764787 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1433/2850] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:10:35.717940 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1a04e-fa05-440c-b5c9-7a545bf05ac6 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:35.718513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1a04e-fa05-440c-b5c9-7a545bf05ac6 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:35.718513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1a04e-fa05-440c-b5c9-7a545bf05ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:35.739226 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-37b1a04e-fa05-440c-b5c9-7a545bf05ac6 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:35.740667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-37b1a04e-fa05-440c-b5c9-7a545bf05ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:35.748072 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1418/2851] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Sep 1 14:10:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:35.760283 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:35.771997 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d6c76-4ae7-4f03-a8bd-2eede49b1f9e service cinder] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:35.777066 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb8db9-6097-4e8d-8281-30082075a4d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:35.777656 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8db9-6097-4e8d-8281-30082075a4d6 service swift] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:35.777975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8db9-6097-4e8d-8281-30082075a4d6 service swift] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:35.778248 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8db9-6097-4e8d-8281-30082075a4d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:35.787042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb8db9-6097-4e8d-8281-30082075a4d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:35.788618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb8db9-6097-4e8d-8281-30082075a4d6 service swift] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:35.796878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1434/2852] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:10:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:10:36.110147 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1400b7-836c-4020-ad62-d7821435c9a4 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:36.110507 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1400b7-836c-4020-ad62-d7821435c9a4 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:36.110850 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1400b7-836c-4020-ad62-d7821435c9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:36.131683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-cb1400b7-836c-4020-ad62-d7821435c9a4 None None] MFA Rules not processed for user `748b8b4771ac498a88fdd98c8a4914ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:36.132949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-cb1400b7-836c-4020-ad62-d7821435c9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:36.150993 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=100 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:36.194496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1419/2853] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:36.516128 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:36.516415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:36.516694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:36.539212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] MFA Rules not processed for user `2dbc06c0703142ba98ffeb72280e1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:36.540444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:36.548063 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1435/2854] 10.222.0.61 () {58 vars in 967 bytes} [Thu Sep 1 14:10:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:36.561377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1400b7-836c-4020-ad62-d7821435c9a4 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:36.574458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1400b7-836c-4020-ad62-d7821435c9a4 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:36.580050 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8219226-b2f1-436e-aa07-e3bce27133dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f29e56690b343269681dfd8caffe001', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:36.580667 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8219226-b2f1-436e-aa07-e3bce27133dc service swift] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:36.580934 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8219226-b2f1-436e-aa07-e3bce27133dc service swift] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:36.581811 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8219226-b2f1-436e-aa07-e3bce27133dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:36.590246 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8219226-b2f1-436e-aa07-e3bce27133dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:36.591999 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8219226-b2f1-436e-aa07-e3bce27133dc service swift] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:36.600395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1420/2855] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Sep 1 14:10:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 14:10:40.665956 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:40.667503 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:40.741609 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ea8b-b631-4380-9c94-d092802ca635 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:40.746249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:40.751123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:40.751123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:40.751123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] PATH_INFO: `/v3/users/1d333e453a8d4df8a1f58d6ffe30fce7` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:40.751123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d333e453a8d4df8a1f58d6ffe30fce7)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:40.751123 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:40.755908 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:41.056863 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d333e453a8d4df8a1f58d6ffe30fce7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:41.063606 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d333e453a8d4df8a1f58d6ffe30fce7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:41.064294 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d333e453a8d4df8a1f58d6ffe30fce7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:41.065006 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d333e453a8d4df8a1f58d6ffe30fce7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:41.169013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d333e453a8d4df8a1f58d6ffe30fce7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:41.188951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1436/2856] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:10:40 2022] DELETE /identity/v3/users/1d333e453a8d4df8a1f58d6ffe30fce7 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:41.193696 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:41.194187 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:41.194313 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:41.213101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:41.214352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:41.274721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1421/2857] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:10:41.284953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:41.286424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:41.345016 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2df988-d316-4b7e-a0f5-45c2a38a607b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:41.348843 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:41.349946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:41.349946 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:41.350191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:41.359836 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:41.361250 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:41.373800 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1437/2858] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:41.786513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:41.786513 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:41.807703 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fca8a9-6a23-46dd-b597-84cbc900179d None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:41.812777 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:41.813532 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:41.813924 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:41.814216 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] PATH_INFO: `/v3/projects/442a8080466045d1ac6d252642b56208` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:41.815941 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442a8080466045d1ac6d252642b56208)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:41.818568 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:41.953539 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442a8080466045d1ac6d252642b56208', 'request_id': 'req-8bcb415e-fba7-4de2-a7b5-7e2539085a50'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:41.961081 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442a8080466045d1ac6d252642b56208', 'request_id': 'req-8bcb415e-fba7-4de2-a7b5-7e2539085a50'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.057569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442a8080466045d1ac6d252642b56208', 'request_id': 'req-8bcb415e-fba7-4de2-a7b5-7e2539085a50'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.078828 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] The token cache is being invalidate because project 442a8080466045d1ac6d252642b56208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:10:42.079254 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.080151 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-8bcb415e-fba7-4de2-a7b5-7e2539085a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.082676 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1422/2859] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:10:41 2022] DELETE /identity/v3/projects/442a8080466045d1ac6d252642b56208 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:42.128663 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:42.130888 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:42.148337 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee8eaf-8f3b-4442-86e5-4eecd66b9788 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:42.230272 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:42.231101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:42.231473 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:42.231854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] PATH_INFO: `/v3/users/392ba34402bb49278ee12c6bf89715d6` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:42.233202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] RBAC: Authorizing `identity:delete_user(user_id=392ba34402bb49278ee12c6bf89715d6)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:42.235535 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:42.509750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:42.696967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '392ba34402bb49278ee12c6bf89715d6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.703772 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '392ba34402bb49278ee12c6bf89715d6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.704468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '392ba34402bb49278ee12c6bf89715d6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.705119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '392ba34402bb49278ee12c6bf89715d6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.786962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '392ba34402bb49278ee12c6bf89715d6'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:42.807470 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1438/2860] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:10:42 2022] DELETE /identity/v3/users/392ba34402bb49278ee12c6bf89715d6 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:42.813937 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:42.814425 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:42.814523 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:42.834558 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:42.835808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:42.871518 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:42.898979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1423/2861] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:42.908696 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:42.910854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:42.968688 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828bcbe-19d6-49fd-9c69-b0dacc99786b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:42.972936 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:42.973539 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:42.979026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:42.979026 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:42.983469 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:42.984904 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:42.994854 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1439/2862] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:43.502055 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:43.503647 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:43.516277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770fb0-314a-4785-90ae-0a279aff6d74 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:43.521160 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:43.521914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:43.522197 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:43.522470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] PATH_INFO: `/v3/projects/bd1cae0cf4124ec392459a4ce7c234bf` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:43.523648 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1cae0cf4124ec392459a4ce7c234bf)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:43.525582 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:43.666792 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1cae0cf4124ec392459a4ce7c234bf', 'request_id': 'req-d27fe275-327d-46e0-8276-ff3cf05c2e28'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:43.672310 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1cae0cf4124ec392459a4ce7c234bf', 'request_id': 'req-d27fe275-327d-46e0-8276-ff3cf05c2e28'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:43.797320 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1cae0cf4124ec392459a4ce7c234bf', 'request_id': 'req-d27fe275-327d-46e0-8276-ff3cf05c2e28'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:43.809631 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] The token cache is being invalidate because project bd1cae0cf4124ec392459a4ce7c234bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:10:43.809957 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:43.810595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:43.812075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1424/2863] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:10:43 2022] DELETE /identity/v3/projects/bd1cae0cf4124ec392459a4ce7c234bf => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:43.816932 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:43.827394 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749f1bf-6028-497e-8a2a-3a81e7dd54db service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:43.889646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:43.890299 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:43.890577 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:43.890849 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] PATH_INFO: `/v3/users/6f938e411eaf43e483532c42d8ccb4fd` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:43.892361 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f938e411eaf43e483532c42d8ccb4fd)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:43.895070 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:44.114298 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f938e411eaf43e483532c42d8ccb4fd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:44.116630 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:44.122327 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f938e411eaf43e483532c42d8ccb4fd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:44.123075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f938e411eaf43e483532c42d8ccb4fd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:44.123693 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f938e411eaf43e483532c42d8ccb4fd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:44.225761 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f938e411eaf43e483532c42d8ccb4fd'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:44.243365 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1440/2864] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:10:43 2022] DELETE /identity/v3/users/6f938e411eaf43e483532c42d8ccb4fd => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:44.725306 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:44.726825 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:44.741099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe275-327d-46e0-8276-ff3cf05c2e28 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:44.803121 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:44.803760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:44.804043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:44.804319 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] PATH_INFO: `/v3/projects/b9f0d3accae145d18e928811c77c29c2` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:44.805456 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f0d3accae145d18e928811c77c29c2)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:44.807389 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:44.946470 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f0d3accae145d18e928811c77c29c2', 'request_id': 'req-c5b6c923-7c8c-41fd-8c00-a1331685cc84'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:44.952279 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f0d3accae145d18e928811c77c29c2', 'request_id': 'req-c5b6c923-7c8c-41fd-8c00-a1331685cc84'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:44.955955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:45.031517 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f0d3accae145d18e928811c77c29c2', 'request_id': 'req-c5b6c923-7c8c-41fd-8c00-a1331685cc84'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.043288 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] The token cache is being invalidate because project b9f0d3accae145d18e928811c77c29c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:10:45.043577 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.044248 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.045760 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1425/2865] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:10:44 2022] DELETE /identity/v3/projects/b9f0d3accae145d18e928811c77c29c2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:45.050438 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:45.051929 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:45.064716 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace9f5-748d-47d2-a13d-c095b43b2062 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:45.125121 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:45.125753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:45.126058 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:45.126332 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] PATH_INFO: `/v3/users/88d4ad64803946d5ac1fcf05813c76fb` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:45.127492 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d4ad64803946d5ac1fcf05813c76fb)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:45.130282 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:45.411965 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d4ad64803946d5ac1fcf05813c76fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.419414 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d4ad64803946d5ac1fcf05813c76fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.420452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d4ad64803946d5ac1fcf05813c76fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.421068 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d4ad64803946d5ac1fcf05813c76fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.552271 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c3ba1f6c-1b58-4824-ac19-fbf66f8b993b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d4ad64803946d5ac1fcf05813c76fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:45.571263 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1441/2866] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:10:45 2022] DELETE /identity/v3/users/88d4ad64803946d5ac1fcf05813c76fb => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:46.108199 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:46.109646 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:46.122685 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6c923-7c8c-41fd-8c00-a1331685cc84 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:46.214416 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dc53a-58db-4121-ab5b-8c0f7f7d8587 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:46.214903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:46.214903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:46.214903 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:46.215169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dc53a-58db-4121-ab5b-8c0f7f7d8587 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:46.215169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dc53a-58db-4121-ab5b-8c0f7f7d8587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:46.215323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] PATH_INFO: `/v3/projects/67811bc54dcc48cb95a94dabcdd2fc07` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:46.215323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67811bc54dcc48cb95a94dabcdd2fc07)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:46.215323 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:46.236689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-d99dc53a-58db-4121-ab5b-8c0f7f7d8587 None None] MFA Rules not processed for user `35383b224a5a46de87c41b49fe1bd037`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:46.239202 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d99dc53a-58db-4121-ab5b-8c0f7f7d8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:46.287996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:46.303382 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1442/2867] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:46.349745 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67811bc54dcc48cb95a94dabcdd2fc07', 'request_id': 'req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:46.358847 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67811bc54dcc48cb95a94dabcdd2fc07', 'request_id': 'req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:46.460427 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67811bc54dcc48cb95a94dabcdd2fc07', 'request_id': 'req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:46.473674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] The token cache is being invalidate because project 67811bc54dcc48cb95a94dabcdd2fc07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:10:46.474034 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:46.474670 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-9b9f4a0e-6f34-4da0-a325-996ecd63bbc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:10:46.476200 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1426/2868] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:10:46 2022] DELETE /identity/v3/projects/67811bc54dcc48cb95a94dabcdd2fc07 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:46.482212 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab5e7e-d465-40b6-af4e-29836161ad89 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:46.482557 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab5e7e-d465-40b6-af4e-29836161ad89 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:46.482848 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab5e7e-d465-40b6-af4e-29836161ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:46.503864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-4fab5e7e-d465-40b6-af4e-29836161ad89 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:46.505336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-4fab5e7e-d465-40b6-af4e-29836161ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:46.568597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1443/2869] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:46.576477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8d35f-577d-4183-9f2c-d1d5ef2ddbca None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:46.576763 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8d35f-577d-4183-9f2c-d1d5ef2ddbca None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:46.577035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8d35f-577d-4183-9f2c-d1d5ef2ddbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:46.598035 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-0de8d35f-577d-4183-9f2c-d1d5ef2ddbca None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:46.600322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-0de8d35f-577d-4183-9f2c-d1d5ef2ddbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:46.607409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1427/2870] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 14:10:46.612109 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab5e7e-d465-40b6-af4e-29836161ad89 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:46.622260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab5e7e-d465-40b6-af4e-29836161ad89 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:46.627811 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:46.628420 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:46.628691 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:46.628960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] PATH_INFO: `/v3/domains` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:46.629608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:10:46.630178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:46.631780 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:46.644355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1444/2871] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:10:46.648712 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8d35f-577d-4183-9f2c-d1d5ef2ddbca None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:46.658179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8d35f-577d-4183-9f2c-d1d5ef2ddbca None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:46.663898 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:46.664510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:46.664793 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:46.665070 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] PATH_INFO: `/v3/projects` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:46.665719 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:46.667721 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:46.811784 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1428/2872] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:10:46 2022] POST /identity/v3/projects => generated 391 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:10:46.813166 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:46.825605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6cd5b-38a6-40b8-b65f-abf43e751462 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:46.893468 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:46.894114 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:46.897873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:46.897873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] PATH_INFO: `/v3/users` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:46.897873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:46.897873 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:47.030521 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1445/2873] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:10:46 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:10:47.035496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:47.046885 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fb9c8-f7e8-47d7-b52b-8b655c3cdc2a admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:47.051946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:47.052585 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:47.052887 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:47.053821 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:47.054569 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:47.056174 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:47.069951 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1429/2874] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:10:47.071112 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:47.081435 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0ed65-47fe-4894-b1ae-e4ec97716472 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:47.086758 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1593cf8-5dab-4392-9baf-2c521e6c5c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:47.087450 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1593cf8-5dab-4392-9baf-2c521e6c5c2a admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:47.087731 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1593cf8-5dab-4392-9baf-2c521e6c5c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:47.088008 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1593cf8-5dab-4392-9baf-2c521e6c5c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:47.088668 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1593cf8-5dab-4392-9baf-2c521e6c5c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:47.090674 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1593cf8-5dab-4392-9baf-2c521e6c5c2a admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:47.105477 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1446/2875] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:10:47.106668 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:47.117835 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45b1fc-c7e3-4ea9-97ed-e708b58de621 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:47.122675 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:47.123343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:47.123606 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:47.123875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] PATH_INFO: `/v3/projects/e0112a0922eb4d85b039063b4e6b9bad/users/7b81f5f915c8435a9817a57efcd98346/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:47.139497 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0112a0922eb4d85b039063b4e6b9bad, user_id=7b81f5f915c8435a9817a57efcd98346, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:47.142096 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:47.220777 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1430/2876] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:10:47 2022] PUT /identity/v3/projects/e0112a0922eb4d85b039063b4e6b9bad/users/7b81f5f915c8435a9817a57efcd98346/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:10:47.233482 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:47.234103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:47.234103 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:47.263987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] MFA Rules not processed for user `7b81f5f915c8435a9817a57efcd98346`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:10:47.265268 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:47.311347 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:47.331867 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1447/2877] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:10:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:10:47.342590 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:10:47.344331 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:10:47.366564 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:47.405343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed98e-bfef-4d8b-b659-ea91af3b4baa admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:10:47.409077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:10:47.410291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:10:47.410574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:10:47.410846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:10:47.419416 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:10:47.420841 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:10:47.430514 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1431/2878] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:10:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:10:57.785882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=6 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:10:58.026206 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=18 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:02.885082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:02.886666 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:02.949723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1c607-6869-459c-b429-9912adcf1c79 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:02.954168 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f69db-2912-4b47-bbb5-a5129b1d4109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:02.954812 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f69db-2912-4b47-bbb5-a5129b1d4109 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:02.955111 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f69db-2912-4b47-bbb5-a5129b1d4109 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:02.955967 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f69db-2912-4b47-bbb5-a5129b1d4109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:02.959097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-363f69db-2912-4b47-bbb5-a5129b1d4109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:02.971184 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f69db-2912-4b47-bbb5-a5129b1d4109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:02.974145 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f69db-2912-4b47-bbb5-a5129b1d4109 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:02.982951 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1448/2879] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:09.462450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:09.528618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84709a40-b75f-4cf1-bf9f-4d0bfdc8f812 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:09.533429 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:09.534099 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:09.534381 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:09.534653 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] PATH_INFO: `/v3/users/7b81f5f915c8435a9817a57efcd98346` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:09.535837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b81f5f915c8435a9817a57efcd98346)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:09.537836 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:09.865077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b81f5f915c8435a9817a57efcd98346'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:09.872437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b81f5f915c8435a9817a57efcd98346'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:09.986046 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b81f5f915c8435a9817a57efcd98346'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:09.986765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b81f5f915c8435a9817a57efcd98346'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:10.000465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b81f5f915c8435a9817a57efcd98346'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:10.006985 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1432/2880] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:11:09 2022] DELETE /identity/v3/users/7b81f5f915c8435a9817a57efcd98346 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:11:10.011707 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:10.012213 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:10.012324 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:10.031739 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:10.033024 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:10.096497 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1449/2881] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:10.106468 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:10.107970 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:10.178583 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5f01-1cd6-4309-b235-f53e79227398 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:10.182333 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3373f00-982d-40fe-aa71-3142fa059ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:10.182950 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3373f00-982d-40fe-aa71-3142fa059ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:10.183233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3373f00-982d-40fe-aa71-3142fa059ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:10.183509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3373f00-982d-40fe-aa71-3142fa059ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:10.193674 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3373f00-982d-40fe-aa71-3142fa059ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:10.195136 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3373f00-982d-40fe-aa71-3142fa059ae4 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:10.204724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1433/2882] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:10.731191 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:10.734554 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:10.747380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3841-7fc6-4cff-80b9-46a2fadc59ca None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:10.752020 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:10.752646 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:10.752922 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:10.753203 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] PATH_INFO: `/v3/projects/e0112a0922eb4d85b039063b4e6b9bad` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:10.755859 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0112a0922eb4d85b039063b4e6b9bad)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:10.757868 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:10.924009 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0112a0922eb4d85b039063b4e6b9bad', 'request_id': 'req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:10.930939 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0112a0922eb4d85b039063b4e6b9bad', 'request_id': 'req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:10.937228 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0112a0922eb4d85b039063b4e6b9bad', 'request_id': 'req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:11.065415 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] The token cache is being invalidate because project e0112a0922eb4d85b039063b4e6b9bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:11:11.066084 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:11.066417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-c474f8bc-d9db-4f27-8f3f-ba5d96c311fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:11:11.067979 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1450/2883] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:11:10 2022] DELETE /identity/v3/projects/e0112a0922eb4d85b039063b4e6b9bad => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:11:11.073048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281131d-c7ff-4a5a-9ef0-902341c0be26 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.073527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281131d-c7ff-4a5a-9ef0-902341c0be26 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.073655 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281131d-c7ff-4a5a-9ef0-902341c0be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.093231 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-f281131d-c7ff-4a5a-9ef0-902341c0be26 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:11.095068 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-f281131d-c7ff-4a5a-9ef0-902341c0be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:11.111548 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:11.152946 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1434/2884] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:11.161793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882957a-673d-40f9-8483-df28bdab1c7d None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.162078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882957a-673d-40f9-8483-df28bdab1c7d None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.162355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882957a-673d-40f9-8483-df28bdab1c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.182417 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c882957a-673d-40f9-8483-df28bdab1c7d None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:11.183604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c882957a-673d-40f9-8483-df28bdab1c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:11.192059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1451/2885] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:11.196574 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281131d-c7ff-4a5a-9ef0-902341c0be26 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:11.206365 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281131d-c7ff-4a5a-9ef0-902341c0be26 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:11.211618 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:11.212846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.213343 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.213651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.214599 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:11:11.215255 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:11.216996 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:11.228642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1435/2886] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:11:11.232987 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882957a-673d-40f9-8483-df28bdab1c7d None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:11.242524 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882957a-673d-40f9-8483-df28bdab1c7d None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:11.247297 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:11.247905 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.248195 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.248489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.249140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:11.251080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:11.386375 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:11.393980 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1452/2887] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:11:11 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:11:11.398165 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:11.408369 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75465049-a322-4f02-95a0-ed164d20debd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:11.465284 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:11.465929 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.466207 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.466478 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.467112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:11.469009 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:11.574987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1436/2888] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:11:11 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:11:11.579105 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:11.590500 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f72e5e-3a9f-4d56-901f-ad765881e93b admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:11.595140 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:11.595733 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.596010 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.596281 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.596925 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:11.599186 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:11.609157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1453/2889] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:11:11.613276 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:11.623579 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e0abc-37a0-4f64-aab9-7357cb602027 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:11.628859 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3f13d-b6d9-46c1-9132-090dd21b7dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:11.629458 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3f13d-b6d9-46c1-9132-090dd21b7dab admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.629744 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3f13d-b6d9-46c1-9132-090dd21b7dab admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.630058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3f13d-b6d9-46c1-9132-090dd21b7dab admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.630979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3f13d-b6d9-46c1-9132-090dd21b7dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:11.632672 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3f13d-b6d9-46c1-9132-090dd21b7dab admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:11.643245 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1437/2890] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:11:11.647953 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:11.658164 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810029c2-9447-464f-83b4-47f8d50f513f admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:11.662882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:11.663531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] REQUEST_METHOD: `PUT` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.663810 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.673317 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] PATH_INFO: `/v3/projects/fb745d3f3bcb44ae9fe9cf80ce054456/users/3b605f7c7e2e44ccaf652fabbbf4080a/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.678659 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb745d3f3bcb44ae9fe9cf80ce054456, user_id=3b605f7c7e2e44ccaf652fabbbf4080a, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:11.681340 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:11.774654 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1454/2891] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:11:11 2022] PUT /identity/v3/projects/fb745d3f3bcb44ae9fe9cf80ce054456/users/3b605f7c7e2e44ccaf652fabbbf4080a/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:11:11.779689 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf072f-5137-46c0-b19c-372280d33f5a None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.779972 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf072f-5137-46c0-b19c-372280d33f5a None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.780243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf072f-5137-46c0-b19c-372280d33f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.799796 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-ecaf072f-5137-46c0-b19c-372280d33f5a None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:11.801036 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-ecaf072f-5137-46c0-b19c-372280d33f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:11.862438 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1438/2892] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:11.872785 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:11.874476 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:11.932178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccccc03-b3c6-4f1c-ad9e-ea49aa5aaa92 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:11.936075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:11.936684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:11.936960 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:11.937227 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:11.946641 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:11.948064 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:11.957724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1455/2893] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:18.119611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c300-21ab-4b0f-b7d3-c8b10bba0d31 None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:18.120091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c300-21ab-4b0f-b7d3-c8b10bba0d31 None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:18.120239 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c300-21ab-4b0f-b7d3-c8b10bba0d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:18.150258 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-dda3c300-21ab-4b0f-b7d3-c8b10bba0d31 None None] MFA Rules not processed for user `3b605f7c7e2e44ccaf652fabbbf4080a`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:18.151464 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-dda3c300-21ab-4b0f-b7d3-c8b10bba0d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:18.169736 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:18.221893 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1439/2894] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:11:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:18.227078 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:18.238383 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb4eef-f88b-4820-8542-767ef4b15644 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:18.243370 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:18.243975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:18.244996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:18.244996 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:18.245291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:18.247402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:18.370721 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1456/2895] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:11:18 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 14:11:18.374770 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3c300-21ab-4b0f-b7d3-c8b10bba0d31 None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:18.387318 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3c300-21ab-4b0f-b7d3-c8b10bba0d31 None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:18.448390 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:18.458378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:18.458378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:18.458378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:18.458378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:18.458378 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:18.637982 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1440/2896] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:11:18 2022] POST /identity/v3/users => generated 291 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:11:18.642077 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:18.653291 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cf259-f21e-4a8f-aeb8-cb11789dde85 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:18.657901 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:18.658395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:18.658669 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:18.658938 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:18.659576 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:18.661165 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:18.673147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1457/2897] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:11:18.677016 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:18.688305 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae7f57-8a86-4b34-ab66-eed53a5a9597 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:18.693686 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:18.694525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:18.694802 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:18.695071 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] PATH_INFO: `/v3/projects/1539856968c543159c1171d48210c517/users/73e8a093b6b4421aad31a5d1ad17506b/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:18.710630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1539856968c543159c1171d48210c517, user_id=73e8a093b6b4421aad31a5d1ad17506b, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:18.716188 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:18.849998 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1441/2898] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:11:18 2022] PUT /identity/v3/projects/1539856968c543159c1171d48210c517/users/73e8a093b6b4421aad31a5d1ad17506b/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:11:18.854080 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:18.865553 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978077df-a54d-4fb7-bb3c-7b64284ca9ba admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:18.926727 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee20123-f4d1-4d57-af47-19e55ad7d207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:18.927350 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20123-f4d1-4d57-af47-19e55ad7d207 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:18.927623 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20123-f4d1-4d57-af47-19e55ad7d207 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:18.927895 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20123-f4d1-4d57-af47-19e55ad7d207 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:18.928542 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee20123-f4d1-4d57-af47-19e55ad7d207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:18.930187 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee20123-f4d1-4d57-af47-19e55ad7d207 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:18.940684 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1458/2899] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:11:18.946533 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:18.957498 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287967a-3ffa-4c29-8255-be6bb2daadbd admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:18.962392 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:18.963155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:18.963506 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:18.963875 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] PATH_INFO: `/v3/projects/1539856968c543159c1171d48210c517/users/73e8a093b6b4421aad31a5d1ad17506b/roles/d1f9f95d9d0841e59b294cb7901942a1` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:18.965906 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1539856968c543159c1171d48210c517, user_id=73e8a093b6b4421aad31a5d1ad17506b, role_id=d1f9f95d9d0841e59b294cb7901942a1)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:18.968697 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:18.979961 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1442/2900] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:11:18 2022] PUT /identity/v3/projects/1539856968c543159c1171d48210c517/users/73e8a093b6b4421aad31a5d1ad17506b/roles/d1f9f95d9d0841e59b294cb7901942a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:11:25.411465 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29172eb-4101-40a0-886c-82a57a4b4c34 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:25.412487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29172eb-4101-40a0-886c-82a57a4b4c34 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:25.412772 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29172eb-4101-40a0-886c-82a57a4b4c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:25.443144 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-b29172eb-4101-40a0-886c-82a57a4b4c34 None None] MFA Rules not processed for user `73e8a093b6b4421aad31a5d1ad17506b`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:25.444387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-b29172eb-4101-40a0-886c-82a57a4b4c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:25.462141 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:25.524760 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1459/2901] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:11:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:25.541525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:25.543487 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:25.628848 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5100e-f39a-4b7a-a6f2-0e4880e60810 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:25.635025 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:25.635884 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:25.636179 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:25.636424 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:25.642102 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:25.696865 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:25.698553 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:25.708637 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1443/2902] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:29.384206 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:29.384697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:29.384799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:29.406424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] MFA Rules not processed for user `3b605f7c7e2e44ccaf652fabbbf4080a`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:11:29.407892 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:29.416529 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1460/2903] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:11:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:11:35.762955 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=24 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:37.114596 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:37.126450 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a72d-2ade-4f15-bc24-d1c917c345d8 service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:37.133527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:37.134177 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:37.134454 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:37.134723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:37.136444 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:37.150409 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:37.151989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:37.163473 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1444/2904] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:39.400583 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=24 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:47.158492 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=6 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:11:50.794753 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:50.796355 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:50.854940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20cdac-8819-4fc5-b5e5-ba494d3b3f86 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:50.858597 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:50.859201 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:50.859475 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:50.859745 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:50.861477 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:50.872512 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:50.873971 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:50.882570 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1461/2905] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:11:51.884123 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:11:51.885816 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:51.990243 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942be50-1e32-4a47-8241-0b6c3463187b service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:11:51.991488 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:11:51.992101 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:11:51.992379 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:11:51.992651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:11:51.994439 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:11:52.007787 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:11:52.009418 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:11:52.021089 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1445/2906] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:11:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:00.879062 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=26 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:02.015677 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=32 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:08.865235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:08.880750 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae95633-e7f3-47c7-a775-9df1bef8ca62 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:08.890097 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:08.890221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:08.890621 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:08.890963 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:08.893260 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:08.905829 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:08.907432 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:08.918348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1462/2907] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:12:09.033817 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:09.035915 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:09.108434 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b21f3-3ade-4021-8da7-aa5479f822d2 service placement] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:09.113102 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:09.113737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:09.114045 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:09.120555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] PATH_INFO: `/v3/users/35383b224a5a46de87c41b49fe1bd037` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:09.120555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35383b224a5a46de87c41b49fe1bd037)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:09.120555 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:09.320169 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35383b224a5a46de87c41b49fe1bd037'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:09.327612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35383b224a5a46de87c41b49fe1bd037'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:09.470657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35383b224a5a46de87c41b49fe1bd037'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:09.471289 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35383b224a5a46de87c41b49fe1bd037'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:09.484175 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-5ce208be-f24c-4460-91e4-446494bb9e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35383b224a5a46de87c41b49fe1bd037'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:09.491542 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1446/2908] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:12:09 2022] DELETE /identity/v3/users/35383b224a5a46de87c41b49fe1bd037 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:12:09.889572 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:09.901602 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf9a5b-071f-411d-be38-1ab91e558119 service placement] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:09.963704 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:09.970402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:09.970402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:09.970402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] PATH_INFO: `/v3/projects/035b87bdd94646d0aed7727e587ab40f` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:09.970402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035b87bdd94646d0aed7727e587ab40f)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:09.970402 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.077793 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035b87bdd94646d0aed7727e587ab40f', 'request_id': 'req-a6b0c6d1-5997-4246-abae-d177314af924'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:10.084618 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035b87bdd94646d0aed7727e587ab40f', 'request_id': 'req-a6b0c6d1-5997-4246-abae-d177314af924'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:10.090489 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035b87bdd94646d0aed7727e587ab40f', 'request_id': 'req-a6b0c6d1-5997-4246-abae-d177314af924'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:10.201724 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] The token cache is being invalidate because project 035b87bdd94646d0aed7727e587ab40f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:12:10.202040 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:10.202657 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-a6b0c6d1-5997-4246-abae-d177314af924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:10.204147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1463/2909] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:12:09 2022] DELETE /identity/v3/projects/035b87bdd94646d0aed7727e587ab40f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:10.209757 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db39e-d72f-4b9b-83aa-7366b48daa3f None None] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.210291 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db39e-d72f-4b9b-83aa-7366b48daa3f None None] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.210400 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db39e-d72f-4b9b-83aa-7366b48daa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.230047 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.auth.core [None req-728db39e-d72f-4b9b-83aa-7366b48daa3f None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:10.231387 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-728db39e-d72f-4b9b-83aa-7366b48daa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:10.290987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1447/2910] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:10.298974 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a8c8e-937c-44e3-a9f2-0d875b53775c None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.299349 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a8c8e-937c-44e3-a9f2-0d875b53775c None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.299742 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a8c8e-937c-44e3-a9f2-0d875b53775c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.321550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-c32a8c8e-937c-44e3-a9f2-0d875b53775c None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:10.322815 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-c32a8c8e-937c-44e3-a9f2-0d875b53775c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:10.331723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1464/2911] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:10.336141 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728db39e-d72f-4b9b-83aa-7366b48daa3f None None] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:10.346364 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728db39e-d72f-4b9b-83aa-7366b48daa3f None None] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:10.351233 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:10.351874 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.352146 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.352417 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] PATH_INFO: `/v3/domains` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.353075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 14:12:10.353372 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:10.355162 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.366949 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1448/2912] 10.222.0.61 () {58 vars in 1153 bytes} [Thu Sep 1 14:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 14:12:10.371678 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a8c8e-937c-44e3-a9f2-0d875b53775c None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:10.382359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a8c8e-937c-44e3-a9f2-0d875b53775c None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:10.387042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:10.387635 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.387909 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.388178 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] PATH_INFO: `/v3/projects` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.388874 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:10.390818 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.536075 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1465/2913] 10.222.0.61 () {60 vars in 1153 bytes} [Thu Sep 1 14:12:10 2022] POST /identity/v3/projects => generated 397 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:12:10.539623 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:10.550629 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca56b4-f0ae-4ae6-94b0-074192289565 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:10.554295 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:10.608765 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:10.609371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.609651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.609964 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] PATH_INFO: `/v3/users` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.610595 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:10.613114 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.757625 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1449/2914] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Sep 1 14:12:10 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 14:12:10.757735 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:10.764830 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67eb7a9-c3d1-44d2-b641-d2868df96faf admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:10.769530 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:10.770194 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.770514 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.770783 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] PATH_INFO: `/v3/roles` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.771430 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:10.773083 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.787100 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1466/2915] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:12:10.788652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:10.800202 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe1c51-0786-4c7e-ae58-a872843593c4 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:10.807901 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:10.808530 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.809090 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.809375 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] PATH_INFO: `/v3/roles` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.809966 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:10.810088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:10.811755 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.811882 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:10.824983 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1450/2916] 10.222.0.61 () {58 vars in 1122 bytes} [Thu Sep 1 14:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 14:12:10.829134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:10.839683 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983db85c-93fc-4983-acde-a38fae68b462 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:10.844630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:10.845419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] REQUEST_METHOD: `PUT` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.845581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.845891 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] PATH_INFO: `/v3/projects/5efdea5c5d9e4d59941010d3699a346b/users/4bcf5f425da44a3985ba92a13d179602/roles/9f2bf5a138d54952a913a69eb59fa144` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.870193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efdea5c5d9e4d59941010d3699a346b, user_id=4bcf5f425da44a3985ba92a13d179602, role_id=9f2bf5a138d54952a913a69eb59fa144)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:10.872288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:10.873496 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.907304 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00811445-c68a-4c0b-8ffa-cbaff1bcb865 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:10.912991 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b3458-c1cd-437a-8659-8350d9d56c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:10.913605 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b3458-c1cd-437a-8659-8350d9d56c0f service nova] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:10.914086 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b3458-c1cd-437a-8659-8350d9d56c0f service nova] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:10.914354 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b3458-c1cd-437a-8659-8350d9d56c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:10.916124 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-452b3458-c1cd-437a-8659-8350d9d56c0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:10.929000 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b3458-c1cd-437a-8659-8350d9d56c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:10.930631 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b3458-c1cd-437a-8659-8350d9d56c0f service nova] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:10.942550 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1467/2917] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:10.991395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1451/2918] 10.222.0.61 () {60 vars in 1483 bytes} [Thu Sep 1 14:12:10 2022] PUT /identity/v3/projects/5efdea5c5d9e4d59941010d3699a346b/users/4bcf5f425da44a3985ba92a13d179602/roles/9f2bf5a138d54952a913a69eb59fa144 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:12:11.004048 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:11.004336 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:11.004612 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:11.036126 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] MFA Rules not processed for user `4bcf5f425da44a3985ba92a13d179602`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:11.037427 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:11.101249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1468/2919] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:12:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:11.113386 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:11.115105 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:11.173695 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483ac92-170d-4876-8807-3af7a90f7abf admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:11.177452 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caa058ab479540848d2d9793fdfe4e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:11.178088 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:11.178395 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:11.178706 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:11.187091 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:11.188597 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:11.198156 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1452/2920] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:13.469132 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:13.470746 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:13.530694 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33faee-4714-45ef-9782-0046c1ec7e48 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:13.534235 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e832fa-2c3a-4476-a217-988fe311b2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:13.534838 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e832fa-2c3a-4476-a217-988fe311b2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:13.535119 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e832fa-2c3a-4476-a217-988fe311b2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:13.535393 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e832fa-2c3a-4476-a217-988fe311b2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:13.537118 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-19e832fa-2c3a-4476-a217-988fe311b2d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:13.600699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e832fa-2c3a-4476-a217-988fe311b2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:13.602221 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e832fa-2c3a-4476-a217-988fe311b2d6 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:13.612531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1469/2921] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:20.492297 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:20.554750 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77600ea8-16b2-4433-9d1d-9c21a36547c0 service cinder] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:20.559301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:20.559926 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:20.560203 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:20.560483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] PATH_INFO: `/v3/users/4bcf5f425da44a3985ba92a13d179602` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:20.561619 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bcf5f425da44a3985ba92a13d179602)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:20.563538 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:20.568642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:20.918612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bcf5f425da44a3985ba92a13d179602'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:20.926469 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bcf5f425da44a3985ba92a13d179602'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:21.024845 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bcf5f425da44a3985ba92a13d179602'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:21.025581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bcf5f425da44a3985ba92a13d179602'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:21.039152 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bcf5f425da44a3985ba92a13d179602'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:21.045111 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1453/2922] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:12:20 2022] DELETE /identity/v3/users/4bcf5f425da44a3985ba92a13d179602 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:21.049608 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] REQUEST_METHOD: `POST` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:21.050157 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:21.050264 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:21.057169 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=100 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:21.068993 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.auth.core [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] MFA Rules not processed for user `2afeb1cf31eb47f9a0c823030df9f4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 14:12:21.070211 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:21.129056 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1470/2923] 10.222.0.61 () {58 vars in 958 bytes} [Thu Sep 1 14:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 14:12:21.139075 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:21.140504 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:21.198078 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f11016-86c2-4cd1-81ed-4f85319fef5b admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:21.201570 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:21.202201 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:21.202483 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:21.202756 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:21.212208 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:21.213630 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:21.223786 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1454/2924] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:21.826302 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:21.833493 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:21.839755 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74821248-b7f7-4052-bda6-bb070ed83e40 None None] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:21.844380 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:21.845013 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:21.845288 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:21.845558 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] PATH_INFO: `/v3/projects/5efdea5c5d9e4d59941010d3699a346b` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:21.847249 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5efdea5c5d9e4d59941010d3699a346b)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:21.848642 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:21.985373 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5efdea5c5d9e4d59941010d3699a346b', 'request_id': 'req-d2a45e6b-801b-4bdc-b34a-37c4149ea998'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:21.992223 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5efdea5c5d9e4d59941010d3699a346b', 'request_id': 'req-d2a45e6b-801b-4bdc-b34a-37c4149ea998'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:21.997906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5efdea5c5d9e4d59941010d3699a346b', 'request_id': 'req-d2a45e6b-801b-4bdc-b34a-37c4149ea998'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:22.099737 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] The token cache is being invalidate because project 5efdea5c5d9e4d59941010d3699a346b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:12:22.100421 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:22.100749 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:12:22.102443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1471/2925] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:12:21 2022] DELETE /identity/v3/projects/5efdea5c5d9e4d59941010d3699a346b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:12:31.218907 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:32.096073 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:12:44.684720 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:12:44.687575 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:44.775028 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0e9c-3c81-4ddb-b190-58456a8680d4 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:12:44.843371 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a91e958d514c0d8a61289bbe3ea63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:12:44.843989 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:12:44.844265 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:12:44.844550 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:12:44.846465 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:12:44.860563 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:12:44.862190 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:12:44.880040 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1455/2926] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:12:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:12:54.869437 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=62 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:00.625037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:00.626616 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:00.685037 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a45e6b-801b-4bdc-b34a-37c4149ea998 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:00.688885 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:00.689483 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:00.689763 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:00.690059 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:00.692321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:00.756230 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:00.757994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:00.768101 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1472/2927] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:13:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:13:08.647990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:08.649604 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:08.665232 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cececc1-5494-47db-9263-9a915c5ba905 service nova] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:08.671627 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:08.672419 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:08.672713 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:08.673012 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] PATH_INFO: `/v3/users/748b8b4771ac498a88fdd98c8a4914ad` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:08.674191 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=748b8b4771ac498a88fdd98c8a4914ad)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:08.676098 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:08.974980 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748b8b4771ac498a88fdd98c8a4914ad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:08.981979 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748b8b4771ac498a88fdd98c8a4914ad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.080185 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748b8b4771ac498a88fdd98c8a4914ad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.080824 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748b8b4771ac498a88fdd98c8a4914ad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.095077 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748b8b4771ac498a88fdd98c8a4914ad'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.101448 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1456/2928] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:08 2022] DELETE /identity/v3/users/748b8b4771ac498a88fdd98c8a4914ad => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:09.375094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:09.386398 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b105a-a70f-4266-889b-6c26711186f2 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:09.443844 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:09.444487 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:09.444759 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:09.445027 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] PATH_INFO: `/v3/projects/9086ba5c610d4954a238d6569ec32f0b` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:09.446233 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9086ba5c610d4954a238d6569ec32f0b)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:09.448151 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:09.595927 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9086ba5c610d4954a238d6569ec32f0b', 'request_id': 'req-2c02c9ae-e17a-40d0-8d64-6014985b059c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.598677 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:09.603321 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9086ba5c610d4954a238d6569ec32f0b', 'request_id': 'req-2c02c9ae-e17a-40d0-8d64-6014985b059c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.609647 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9086ba5c610d4954a238d6569ec32f0b', 'request_id': 'req-2c02c9ae-e17a-40d0-8d64-6014985b059c'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.681491 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] The token cache is being invalidate because project 9086ba5c610d4954a238d6569ec32f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:09.681791 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.682433 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:09.683994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1473/2929] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:09 2022] DELETE /identity/v3/projects/9086ba5c610d4954a238d6569ec32f0b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:09.687987 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:09.689432 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:09.703109 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf110e7-41f0-4d07-ae7c-a4932bdfae42 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:09.759742 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:09.760377 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:09.760657 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:09.760933 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] PATH_INFO: `/v3/users/920218bc8feb4ff88c2a26febf175360` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:09.762112 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920218bc8feb4ff88c2a26febf175360)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:09.764018 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:10.050442 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920218bc8feb4ff88c2a26febf175360'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.057737 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920218bc8feb4ff88c2a26febf175360'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.138896 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920218bc8feb4ff88c2a26febf175360'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.139534 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920218bc8feb4ff88c2a26febf175360'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.148919 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:10.153490 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920218bc8feb4ff88c2a26febf175360'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.160502 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1457/2930] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:13:09 2022] DELETE /identity/v3/users/920218bc8feb4ff88c2a26febf175360 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:10.465864 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:13:10.467310 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:13:10.482042 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c9ae-e17a-40d0-8d64-6014985b059c admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:13:10.542531 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:13:10.543163 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:13:10.543444 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:13:10.543777 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] PATH_INFO: `/v3/projects/9179b845c5254eeb9d47970a15b462d2` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:13:10.544906 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9179b845c5254eeb9d47970a15b462d2)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:13:10.546866 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:13:10.732878 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9179b845c5254eeb9d47970a15b462d2', 'request_id': 'req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.739814 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9179b845c5254eeb9d47970a15b462d2', 'request_id': 'req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.745449 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9179b845c5254eeb9d47970a15b462d2', 'request_id': 'req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.899035 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] The token cache is being invalidate because project 9179b845c5254eeb9d47970a15b462d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:13:10.899359 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.900022 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:13:10.901994 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1474/2931] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:13:10 2022] DELETE /identity/v3/projects/9179b845c5254eeb9d47970a15b462d2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:13:20.578660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:13:20.895172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:22.032509 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:22.033956 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:22.106897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf79665-abd7-4bcb-a582-00805572a8d2 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:22.166106 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:22.166724 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:22.167004 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:22.167275 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:22.169060 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:22.180651 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:22.182277 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:22.192914 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1458/2932] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:23.741852 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:23.743478 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:23.802697 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eac267-c1b3-4c3e-ad4a-f97c17be45ca admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:23.806303 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:23.806897 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:23.807176 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:23.807443 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:23.809155 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:23.819094 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:23.820502 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:23.828348 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1475/2933] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:23.917462 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:14:23.927362 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad3682-f2ef-4945-bf8a-62cd018a5ca0 service placement] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:14:23.930921 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:14:23.931510 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:14:23.931782 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:14:23.932048 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:14:23.933967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:14:24.003938 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:14:24.005352 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:14:24.016552 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1459/2934] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:14:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:14:33.824769 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=26 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:14:34.011611 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=92 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:15:54.733699 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:54.735200 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:54.746454 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ce2a-fd01-4d75-bd73-0f70b2c66273 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:54.750305 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:54.750940 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:54.751218 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:54.751488 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:54.753395 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:54.767943 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:54.769405 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:54.781266 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1476/2935] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:15:57.149120 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:15:57.158345 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3743b54-1b86-4bfe-a94a-26549a94256e service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:15:57.164715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c427a987ba48878873734ecf0f86e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:15:57.164715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:15:57.164715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:15:57.164715 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:15:57.166344 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:15:57.181477 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:15:57.182986 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:15:57.194837 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1460/2936] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:15:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 14:16:03.715452 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:03.716975 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:03.734239 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f833-5451-4a63-8ac2-c5c1c0152710 service neutron] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:03.739799 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:03.740403 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] REQUEST_METHOD: `GET` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:03.740681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:03.740955 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:03.742824 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:03.755448 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:03.757082 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:03.768689 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1477/2937] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:16:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:16:07.189967 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=6 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:07.626808 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:07.628263 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:07.694193 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5477b4d-9006-40cc-bed1-70b79c2882f9 service neutron] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:07.699294 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:07.699928 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:07.700205 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:07.700474 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] PATH_INFO: `/v3/users/3b605f7c7e2e44ccaf652fabbbf4080a` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:07.701601 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b605f7c7e2e44ccaf652fabbbf4080a)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:07.703527 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:07.896652 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b605f7c7e2e44ccaf652fabbbf4080a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:07.903525 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b605f7c7e2e44ccaf652fabbbf4080a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.093738 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b605f7c7e2e44ccaf652fabbbf4080a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.094412 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b605f7c7e2e44ccaf652fabbbf4080a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.107897 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b605f7c7e2e44ccaf652fabbbf4080a'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.123322 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1461/2938] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:16:07 2022] DELETE /identity/v3/users/3b605f7c7e2e44ccaf652fabbbf4080a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:08.393238 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:08.410188 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba8c4-b495-4e68-b46f-0ea8dd3ac0e6 service placement] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:08.477801 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:08.478547 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:08.478857 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:08.479172 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] PATH_INFO: `/v3/projects/fb745d3f3bcb44ae9fe9cf80ce054456` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:08.480406 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb745d3f3bcb44ae9fe9cf80ce054456)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:08.482419 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:08.593424 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb745d3f3bcb44ae9fe9cf80ce054456', 'request_id': 'req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.599686 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb745d3f3bcb44ae9fe9cf80ce054456', 'request_id': 'req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.605143 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb745d3f3bcb44ae9fe9cf80ce054456', 'request_id': 'req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.707607 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] The token cache is being invalidate because project fb745d3f3bcb44ae9fe9cf80ce054456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:16:08.707962 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.708723 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:08.710490 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1478/2939] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:16:08 2022] DELETE /identity/v3/projects/fb745d3f3bcb44ae9fe9cf80ce054456 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:08.714861 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:08.716382 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:08.729642 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2feb6-246a-49e6-99b7-60c1727b452c admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:08.787665 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:08.788301 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:08.788581 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:08.788966 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] PATH_INFO: `/v3/users/73e8a093b6b4421aad31a5d1ad17506b` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:08.790155 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e8a093b6b4421aad31a5d1ad17506b)` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:08.792058 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:08.815024 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:09.013134 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e8a093b6b4421aad31a5d1ad17506b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.019877 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e8a093b6b4421aad31a5d1ad17506b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.135522 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e8a093b6b4421aad31a5d1ad17506b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.136043 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e8a093b6b4421aad31a5d1ad17506b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.149061 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.notifications [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e8a093b6b4421aad31a5d1ad17506b'} {{(pid=88134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.155508 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1462/2940] 10.222.0.61 () {58 vars in 1224 bytes} [Thu Sep 1 14:16:08 2022] DELETE /identity/v3/users/73e8a093b6b4421aad31a5d1ad17506b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 14:16:09.483604 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Authenticating user token {{(pid=88133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:16:09.489387 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.fernet_utils [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:16:09.499374 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25a236-fe3c-4eb2-8fdb-8a8e2ef712e4 admin admin] Validating token access rules against request {{(pid=88133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:16:09.559629 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afeb1cf31eb47f9a0c823030df9f4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c0369ad3194a3b8c268accfe2208d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:16:09.560173 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:16:09.560564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:16:09.560842 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] PATH_INFO: `/v3/projects/1539856968c543159c1171d48210c517` {{(pid=88133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:16:09.561998 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1539856968c543159c1171d48210c517)` {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:16:09.564005 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] RBAC: Authorization granted {{(pid=88133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:16:09.570564 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:09.734104 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1539856968c543159c1171d48210c517', 'request_id': 'req-8ffd9e4d-3daa-418b-a54c-a751afb479e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.740207 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1539856968c543159c1171d48210c517', 'request_id': 'req-8ffd9e4d-3daa-418b-a54c-a751afb479e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.745586 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1539856968c543159c1171d48210c517', 'request_id': 'req-8ffd9e4d-3daa-418b-a54c-a751afb479e7'} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.860789 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] The token cache is being invalidate because project 1539856968c543159c1171d48210c517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 14:16:09.861147 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.881681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG keystone.notifications [None req-8ffd9e4d-3daa-418b-a54c-a751afb479e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 14:16:09.881681 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: [pid: 88133|app: 0|req: 1479/2941] 10.222.0.61 () {58 vars in 1233 bytes} [Thu Sep 1 14:16:09 2022] DELETE /identity/v3/projects/1539856968c543159c1171d48210c517 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 14:16:19.181054 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:16:20.150803 dv-cinder-852210-7-iscsi devstack@keystone.service[88133]: DEBUG dbcounter [-] [88133] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 14:19:51.888660 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Authenticating user token {{(pid=88134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 14:19:51.890244 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:19:52.032459 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eacc3e-47e8-43ce-a546-25c99d0bba80 admin admin] Validating token access rules against request {{(pid=88134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 14:19:52.089846 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67652a76-8821-4545-92aa-d5bb9db07291 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9d9a39675444fd9929465b27fb83cb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a745287849ba435e8bd1b543d58d9307', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 14:19:52.090446 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652a76-8821-4545-92aa-d5bb9db07291 service placement] REQUEST_METHOD: `GET` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 14:19:52.090723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652a76-8821-4545-92aa-d5bb9db07291 service placement] SCRIPT_NAME: `/identity` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 14:19:52.090990 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652a76-8821-4545-92aa-d5bb9db07291 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 14:19:52.092723 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.fernet_utils [None req-67652a76-8821-4545-92aa-d5bb9db07291 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 14:19:52.104056 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67652a76-8821-4545-92aa-d5bb9db07291 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 14:19:52.105612 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67652a76-8821-4545-92aa-d5bb9db07291 service placement] RBAC: Authorization granted {{(pid=88134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 14:19:52.115413 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: [pid: 88134|app: 0|req: 1463/2942] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Sep 1 14:19:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 14:20:02.111349 dv-cinder-852210-7-iscsi devstack@keystone.service[88134]: DEBUG dbcounter [-] [88134] Writing DB stats keystone:SELECT=89 {{(pid=88134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}